[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:31:45 dist-test-slave-tm1g krb5kdc[30637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:31:45 dist-test-slave-tm1g krb5kdc[30637](info): set up 2 sockets
Aug 10 19:31:45 dist-test-slave-tm1g krb5kdc[30637](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 19:31:46.359704 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.010s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:31:46 dist-test-slave-tm1g krb5kdc[30637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:31:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:31:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:31:46.578763 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39673
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:39673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:46.929072 30652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:46.929461 30652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:46.929625 30652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:46.937070 30652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:31:46.937234 30652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:46.937350 30652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:46.937444 30652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:31:46.937528 30652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:31:46.948522 30652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:39673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:46.950078 30652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:46.951575 30652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:46.965493 30661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:46.965512 30659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:46.965534 30657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:46.965993 30652 server_base.cc:1034] running on GCE node
I20260810 19:31:46.966965 30652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:46.968568 30652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:46.969825 30652 hybrid_clock.cc:648] HybridClock initialized: now 1786390306969711 us; error 103 us; skew 500 ppm
I20260810 19:31:46.970371 30652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:46 dist-test-slave-tm1g krb5kdc[30637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:46.980980 30652 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:31:46.982865 30652 webserver.cc:460] Webserver started at http://127.29.233.190:36603/ using document root <none> and password file <none>
I20260810 19:31:46.983664 30652 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:46.983815 30652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:46.984149 30652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:46.987596 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "57221d76f0344793ba3a6b0f29ff6d82"
format_stamp: "Formatted at 2026-08-10 19:31:46 on dist-test-slave-tm1g"
server_key: "732a09b6ba3d21645e77ef9d28a4ad35"
server_key_iv: "cccef26c2a012a43f6a94bb8c7c23fcc"
server_key_version: "encryptionkey@0"
I20260810 19:31:46.988435 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "57221d76f0344793ba3a6b0f29ff6d82"
format_stamp: "Formatted at 2026-08-10 19:31:46 on dist-test-slave-tm1g"
server_key: "732a09b6ba3d21645e77ef9d28a4ad35"
server_key_iv: "cccef26c2a012a43f6a94bb8c7c23fcc"
server_key_version: "encryptionkey@0"
I20260810 19:31:46.994516 30652 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 19:31:46.998550 30669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:47.000072 30652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:31:47.000289 30652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "57221d76f0344793ba3a6b0f29ff6d82"
format_stamp: "Formatted at 2026-08-10 19:31:46 on dist-test-slave-tm1g"
server_key: "732a09b6ba3d21645e77ef9d28a4ad35"
server_key_iv: "cccef26c2a012a43f6a94bb8c7c23fcc"
server_key_version: "encryptionkey@0"
I20260810 19:31:47.000531 30652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:47.016660 30652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:47.021502 30652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:47.021819 30652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:47.041715 30652 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:39673
I20260810 19:31:47.041736 30728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:39673 every 8 connection(s)
I20260810 19:31:47.043300 30652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:31:47.047446 30729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:47.051170 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 30652
I20260810 19:31:47.051524 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:31:47.054144 30630 external_mini_cluster.cc:1428] Setting key 5900239c90170b4e745dc5b7028e871f
I20260810 19:31:47.060702 30729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82: Bootstrap starting.
I20260810 19:31:47.064095 30729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:47.065158 30729 log.cc:824] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:47.068151 30729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82: No bootstrap required, opened a new log
I20260810 19:31:47.078739 30729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57221d76f0344793ba3a6b0f29ff6d82" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39673 } }
I20260810 19:31:47.079231 30729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:47.079365 30729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57221d76f0344793ba3a6b0f29ff6d82, State: Initialized, Role: FOLLOWER
I20260810 19:31:47.080154 30729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57221d76f0344793ba3a6b0f29ff6d82" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39673 } }
I20260810 19:31:47.080461 30729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:31:47.080622 30729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:31:47.080796 30729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:47.083359 30729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57221d76f0344793ba3a6b0f29ff6d82" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39673 } }
I20260810 19:31:47.083889 30729 leader_election.cc:302] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57221d76f0344793ba3a6b0f29ff6d82; no voters:
I20260810 19:31:47.084539 30729 leader_election.cc:288] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:31:47.084755 30734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 1 FOLLOWER]: Leader election won for term 1
Aug 10 19:31:47 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:47.086625 30734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [term 1 LEADER]: Becoming Leader. State: Replica: 57221d76f0344793ba3a6b0f29ff6d82, State: Running, Role: LEADER
I20260810 19:31:47.087123 30729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:31:47.087242 30734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57221d76f0344793ba3a6b0f29ff6d82" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39673 } }
I20260810 19:31:47.090688 30735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57221d76f0344793ba3a6b0f29ff6d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57221d76f0344793ba3a6b0f29ff6d82" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39673 } } }
I20260810 19:31:47.091148 30735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [sys.catalog]: This master's current role is: LEADER
I20260810 19:31:47.093076 30736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57221d76f0344793ba3a6b0f29ff6d82. Latest consensus state: current_term: 1 leader_uuid: "57221d76f0344793ba3a6b0f29ff6d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57221d76f0344793ba3a6b0f29ff6d82" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39673 } } }
I20260810 19:31:47.093560 30736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82 [sys.catalog]: This master's current role is: LEADER
I20260810 19:31:47.105306 30740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:31:47.105495 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:47.060417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.29.233.190:39673)
0810 19:31:47.061383 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:47.061397 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:47.061450 (+ 53us) server_negotiation.cc:407] Connection header received
0810 19:31:47.070994 (+ 9544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:47.071021 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:47.071554 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:47.072014 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:47.074932 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:47.077080 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:47.078937 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:47.080044 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:47.080251 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:47.087578 (+ 7327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:47.087590 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:47.087610 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:47.087665 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:47.093075 (+ 5410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:47.094746 (+ 1671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:47.094760 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:47.094776 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:47.096264 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:47.097094 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:47.097102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:47.097108 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:47.098355 (+ 1247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:47.102039 (+ 3684us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:47.102447 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 19:31:47.103072 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":956,"threads_started":1}
I20260810 19:31:47.111805 30740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:31:47.121520 30740 catalog_manager.cc:1347] Generated new cluster ID: 14ce58dda0d64378950d1741f6d249c2
I20260810 19:31:47.121655 30740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:31:47.161253 30740 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:31:47.163229 30740 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:31:47.176878 30740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57221d76f0344793ba3a6b0f29ff6d82: Generated new TSK 0
I20260810 19:31:47.178135 30740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:31:47.243427 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39673
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:47.596765 30759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:47.597076 30759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:47.597175 30759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:47.604815 30759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:47.604967 30759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:47.605103 30759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:31:47.615432 30759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39673
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:47.616812 30759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:47.618180 30759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:47.633411 30765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:47.633438 30764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:47.633664 30767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:47.634287 30759 server_base.cc:1034] running on GCE node
I20260810 19:31:47.635061 30759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:47.636093 30759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:47.637370 30759 hybrid_clock.cc:648] HybridClock initialized: now 1786390307637282 us; error 47 us; skew 500 ppm
I20260810 19:31:47.637737 30759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:47 dist-test-slave-tm1g krb5kdc[30637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:47.641141 30759 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:31:47.642830 30759 webserver.cc:460] Webserver started at http://127.29.233.129:33917/ using document root <none> and password file <none>
I20260810 19:31:47.643582 30759 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:47.643715 30759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:47.644054 30759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:47.647338 30759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "472b49ee3bb14446a9f7cc541063b6ee"
format_stamp: "Formatted at 2026-08-10 19:31:47 on dist-test-slave-tm1g"
server_key: "4a2a3f0142f062728325521c8f22a45e"
server_key_iv: "b9dbec033095b643ce573dff8ba35ed8"
server_key_version: "encryptionkey@0"
I20260810 19:31:47.648161 30759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "472b49ee3bb14446a9f7cc541063b6ee"
format_stamp: "Formatted at 2026-08-10 19:31:47 on dist-test-slave-tm1g"
server_key: "4a2a3f0142f062728325521c8f22a45e"
server_key_iv: "b9dbec033095b643ce573dff8ba35ed8"
server_key_version: "encryptionkey@0"
I20260810 19:31:47.653877 30759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:31:47.657819 30775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:47.659466 30759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:31:47.659657 30759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "472b49ee3bb14446a9f7cc541063b6ee"
format_stamp: "Formatted at 2026-08-10 19:31:47 on dist-test-slave-tm1g"
server_key: "4a2a3f0142f062728325521c8f22a45e"
server_key_iv: "b9dbec033095b643ce573dff8ba35ed8"
server_key_version: "encryptionkey@0"
I20260810 19:31:47.659881 30759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:47.673146 30759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:47.677772 30759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:47.678100 30759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:47.679212 30759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:31:47.680807 30759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:47.680912 30759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:47.681061 30759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:47.681140 30759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:47.716956 30759 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:36753
I20260810 19:31:47.717007 30901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:36753 every 8 connection(s)
I20260810 19:31:47.718473 30759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:31:47.726648 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 30759
I20260810 19:31:47.726902 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:31:47.727741 30630 external_mini_cluster.cc:1428] Setting key 6000152b68da4858a90f7836a5088e74
Aug 10 19:31:47 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:47.744361 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:47.721398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:34465 (local address 127.29.233.190:39673)
0810 19:31:47.721921 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:47.721930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:47.723006 (+ 1076us) server_negotiation.cc:407] Connection header received
0810 19:31:47.724915 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:47.724920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:47.724995 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:47.725102 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:47.727029 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:47.728185 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:47.730577 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:47.731359 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:47.731522 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:47.736107 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:47.736117 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:47.736124 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:47.736178 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:47.739450 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:47.740688 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:47.740722 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:47.740728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:47.740913 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:47.741582 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:47.741589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:47.741593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:47.742040 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:47.742207 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:47.743430 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0810 19:31:47.743834 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":227,"threads_started":1}
I20260810 19:31:47.745687 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:47.722007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39673 (local address 127.29.233.129:34465)
0810 19:31:47.722806 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:47.722863 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:47.724597 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:47.725338 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:47.725348 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:47.725940 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:47.726822 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:47.726844 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:47.728365 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:47.728370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:47.730405 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:47.730427 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:47.731750 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:47.731759 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:47.732137 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:47.733051 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:47.733084 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:47.735791 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:47.739665 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:47.739679 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:47.739701 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:47.740524 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:47.741066 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:47.741071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:47.741091 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:47.741409 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:47.742207 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:47.742217 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:47.742829 (+ 612us) client_negotiation.cc:769] Sending connection context
0810 19:31:47.743219 (+ 390us) client_negotiation.cc:240] Negotiation successful
0810 19:31:47.743841 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":347,"threads_started":1}
I20260810 19:31:47.748775 30902 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39673
I20260810 19:31:47.749176 30902 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:47.750334 30902 heartbeater.cc:511] Master 127.29.233.190:39673 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
I20260810 19:31:47.752914 30687 ts_manager.cc:194] Registered new tserver with Master: 472b49ee3bb14446a9f7cc541063b6ee (127.29.233.129:36753)
I20260810 19:31:47.754599 30687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:34465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:31:47.782253 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39673
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:48.132233 30908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:48.132555 30908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:48.132727 30908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:48.140277 30908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:48.140407 30908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:48.140592 30908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:31:48.152055 30908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39673
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:48.153527 30908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:48.155053 30908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:48.169950 30916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:48.170162 30914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:48.170544 30913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:48.171181 30908 server_base.cc:1034] running on GCE node
I20260810 19:31:48.171921 30908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:48.173115 30908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:48.174453 30908 hybrid_clock.cc:648] HybridClock initialized: now 1786390308174382 us; error 35 us; skew 500 ppm
I20260810 19:31:48.174818 30908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:48.178223 30908 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:31:48.180116 30908 webserver.cc:460] Webserver started at http://127.29.233.130:40471/ using document root <none> and password file <none>
I20260810 19:31:48.180895 30908 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:48.181005 30908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:48.181298 30908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:48.184423 30908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "24592aedd8474dc58a4b54ff10eb5516"
format_stamp: "Formatted at 2026-08-10 19:31:48 on dist-test-slave-tm1g"
server_key: "fe787b310318b410ccd6178e74157604"
server_key_iv: "ade7e6ffff246db6012423ea2e763c8c"
server_key_version: "encryptionkey@0"
I20260810 19:31:48.185192 30908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "24592aedd8474dc58a4b54ff10eb5516"
format_stamp: "Formatted at 2026-08-10 19:31:48 on dist-test-slave-tm1g"
server_key: "fe787b310318b410ccd6178e74157604"
server_key_iv: "ade7e6ffff246db6012423ea2e763c8c"
server_key_version: "encryptionkey@0"
I20260810 19:31:48.190896 30908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 19:31:48.194689 30924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:48.196185 30908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:31:48.196386 30908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "24592aedd8474dc58a4b54ff10eb5516"
format_stamp: "Formatted at 2026-08-10 19:31:48 on dist-test-slave-tm1g"
server_key: "fe787b310318b410ccd6178e74157604"
server_key_iv: "ade7e6ffff246db6012423ea2e763c8c"
server_key_version: "encryptionkey@0"
I20260810 19:31:48.196621 30908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:48.229398 30908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:48.233927 30908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:48.234256 30908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:48.235383 30908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:31:48.236853 30908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:48.236936 30908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:48.237119 30908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:48.237187 30908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:48.274075 30908 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:42435
I20260810 19:31:48.274166 31060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:42435 every 8 connection(s)
I20260810 19:31:48.275542 30908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:31:48.285835 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 30908
I20260810 19:31:48.286062 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:31:48.286662 30630 external_mini_cluster.cc:1428] Setting key d452511b29329e3ae6fc3da45e3f5c2e
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:48.299685 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:48.278262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:43121 (local address 127.29.233.190:39673)
0810 19:31:48.278791 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:48.278796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:48.279792 (+ 996us) server_negotiation.cc:407] Connection header received
0810 19:31:48.281909 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:48.281915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:48.282013 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:48.282135 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:48.283740 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.284498 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.286563 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.287160 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.287292 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:48.291586 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:48.291595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:48.291603 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:48.291654 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:48.295872 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.296971 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.296976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.296978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.297126 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.297728 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.297732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.297735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.298047 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:48.298200 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:48.299193 (+ 993us) server_negotiation.cc:299] Negotiation successful
0810 19:31:48.299402 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":248,"threads_started":1}
I20260810 19:31:48.301302 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:48.278900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39673 (local address 127.29.233.130:43121)
0810 19:31:48.279617 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:48.279667 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:48.281610 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:48.282421 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:48.282430 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:48.282980 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:48.283566 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:48.283584 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.284668 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.284672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:48.286416 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:48.286431 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.287460 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.287469 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:48.287752 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:48.288583 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:48.288617 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:48.291255 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:48.296066 (+ 4811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:48.296074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:48.296091 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:48.296795 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:48.297251 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:48.297258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:48.297262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:48.297565 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:48.298187 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:48.298198 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:48.298732 (+ 534us) client_negotiation.cc:769] Sending connection context
0810 19:31:48.299066 (+ 334us) client_negotiation.cc:240] Negotiation successful
0810 19:31:48.299693 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":287,"threads_started":1}
I20260810 19:31:48.303020 31061 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39673
I20260810 19:31:48.303404 31061 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:48.304323 31061 heartbeater.cc:511] Master 127.29.233.190:39673 requested a full tablet report, sending...
I20260810 19:31:48.306167 30687 ts_manager.cc:194] Registered new tserver with Master: 24592aedd8474dc58a4b54ff10eb5516 (127.29.233.130:42435)
I20260810 19:31:48.307186 30687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:43121
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:31:48.342927 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39673
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:48.696002 31069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:48.696326 31069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:48.696540 31069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:48.704272 31069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:48.704408 31069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:48.704615 31069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:31:48.714936 31069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39673
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:48.716755 31069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:48.718637 31069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:48.734787 31075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:48.734712 31079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:48.734761 31076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:48.735630 31069 server_base.cc:1034] running on GCE node
I20260810 19:31:48.736310 31069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:48.737347 31069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:48.738574 31069 hybrid_clock.cc:648] HybridClock initialized: now 1786390308738522 us; error 46 us; skew 500 ppm
I20260810 19:31:48.739094 31069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:48.742522 31069 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:31:48.743997 31069 webserver.cc:460] Webserver started at http://127.29.233.131:44093/ using document root <none> and password file <none>
I20260810 19:31:48.744684 31069 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:48.744793 31069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:48.745064 31069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:48.748052 31069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "6da3ede55c2345b5b9caafc6897349d9"
format_stamp: "Formatted at 2026-08-10 19:31:48 on dist-test-slave-tm1g"
server_key: "fdd1254cbe64e5633eaaf8e74fcb98bb"
server_key_iv: "c43590659c073676f92300e02db1593c"
server_key_version: "encryptionkey@0"
I20260810 19:31:48.748783 31069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "6da3ede55c2345b5b9caafc6897349d9"
format_stamp: "Formatted at 2026-08-10 19:31:48 on dist-test-slave-tm1g"
server_key: "fdd1254cbe64e5633eaaf8e74fcb98bb"
server_key_iv: "c43590659c073676f92300e02db1593c"
server_key_version: "encryptionkey@0"
I20260810 19:31:48.754067 31069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:31:48.757619 31089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:48.758219 30902 heartbeater.cc:503] Master 127.29.233.190:39673 was elected leader, sending a full tablet report...
I20260810 19:31:48.759284 31069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:31:48.759464 31069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "6da3ede55c2345b5b9caafc6897349d9"
format_stamp: "Formatted at 2026-08-10 19:31:48 on dist-test-slave-tm1g"
server_key: "fdd1254cbe64e5633eaaf8e74fcb98bb"
server_key_iv: "c43590659c073676f92300e02db1593c"
server_key_version: "encryptionkey@0"
I20260810 19:31:48.759651 31069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:48.787463 31069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:48.792080 31069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:48.792433 31069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:48.793696 31069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:31:48.795281 31069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:48.795358 31069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:48.795516 31069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:48.795568 31069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:48.832417 31069 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:34437
I20260810 19:31:48.832484 31229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:34437 every 8 connection(s)
I20260810 19:31:48.833806 31069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:31:48.836306 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 31069
I20260810 19:31:48.836544 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:31:48.837162 30630 external_mini_cluster.cc:1428] Setting key d7fb0f66944ecf491480d2cd65e1b291
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:48.856675 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:48.836772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:35563 (local address 127.29.233.190:39673)
0810 19:31:48.837304 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:48.837309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:48.838669 (+ 1360us) server_negotiation.cc:407] Connection header received
0810 19:31:48.840850 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:48.840854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:48.840919 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:48.841014 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:48.842729 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.843571 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.845179 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.845630 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.845745 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:48.850167 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:48.850172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:48.850176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:48.850214 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:48.853323 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.854168 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.854178 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.854180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.854351 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.854824 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.854828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.854832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.855123 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:48.855295 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:48.856043 (+ 748us) server_negotiation.cc:299] Negotiation successful
0810 19:31:48.856274 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":264,"threads_started":1}
I20260810 19:31:48.857771 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:48.837488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39673 (local address 127.29.233.131:35563)
0810 19:31:48.838520 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:48.838561 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:48.840559 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:48.841326 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:48.841335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:48.841885 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:48.842560 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:48.842580 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.843728 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.843732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:48.845021 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:48.845034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.846042 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.846063 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:48.846532 (+ 469us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:48.847490 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:48.847526 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:48.849907 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:48.853515 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:48.853526 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:48.853545 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:48.854029 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:48.854473 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:48.854478 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:48.854482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:48.854699 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:48.855264 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:48.855273 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:48.855656 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 19:31:48.855944 (+ 288us) client_negotiation.cc:240] Negotiation successful
0810 19:31:48.856533 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":371,"threads_started":1}
I20260810 19:31:48.859302 31230 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39673
I20260810 19:31:48.859726 31230 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:48.860530 31230 heartbeater.cc:511] Master 127.29.233.190:39673 requested a full tablet report, sending...
I20260810 19:31:48.862058 30687 ts_manager.cc:194] Registered new tserver with Master: 6da3ede55c2345b5b9caafc6897349d9 (127.29.233.131:34437)
I20260810 19:31:48.862983 30687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:35563
I20260810 19:31:48.866400 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:31:48.922022 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:48.905223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.29.233.129:36753)
0810 19:31:48.906013 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:48.906039 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:48.906480 (+ 441us) server_negotiation.cc:407] Connection header received
0810 19:31:48.906713 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:48.906721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:48.906955 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:48.907138 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:48.908714 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.909813 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.911253 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.911856 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.912000 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:48.916165 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:48.916176 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:48.916191 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:48.916226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:48.919128 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.919809 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.919816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.919818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.919945 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.920352 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.920356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.920358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.920721 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:48.921050 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:48.921369 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 19:31:48.921659 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":384,"threads_started":1}
W20260810 19:31:48.923353 30806 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35136
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 19:31:48 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:31:48.976042 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:48.961125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.29.233.129:36753)
0810 19:31:48.961357 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:48.961366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:48.961557 (+ 191us) server_negotiation.cc:407] Connection header received
0810 19:31:48.961710 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:48.961714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:48.961784 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:48.961874 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:48.963701 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.964662 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.966054 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.966584 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.966696 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:48.970743 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:48.970748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:48.970752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:48.970789 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:48.973468 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.974093 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.974097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.974099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.974226 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.974640 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.974644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.974646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.974912 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:48.975093 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:48.975446 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 19:31:48.975694 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 19:31:48.977864 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 30759
I20260810 19:31:48.991958 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:48.978906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35162 (local address 127.29.233.129:36753)
0810 19:31:48.979079 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:48.979083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:48.979520 (+ 437us) server_negotiation.cc:407] Connection header received
0810 19:31:48.979645 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:48.979649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:48.979709 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:48.979797 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:48.981079 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.981819 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.983375 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:48.983877 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:48.983986 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:48.986650 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:48.986656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:48.986659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:48.986694 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:48.989475 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.990160 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.990164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.990166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.990334 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:48.990726 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:48.990729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:48.990730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:48.990996 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:48.991171 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:48.991505 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 19:31:48.991675 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 19:31:48.992771 30897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:49.081655 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 30759
I20260810 19:31:49.102365 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 30908
Aug 10 19:31:49 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:31:49.121129 31248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:49.104145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.29.233.130:42435)
0810 19:31:49.104841 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:49.104849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:49.104878 (+ 29us) server_negotiation.cc:407] Connection header received
0810 19:31:49.104952 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:49.104959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:49.105186 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:49.105386 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:49.106727 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:49.107884 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:49.109205 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:49.109874 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:49.109985 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:49.114718 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:49.114734 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:49.114750 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:49.114788 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:49.117842 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:49.118784 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:49.118792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:49.118794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:49.118960 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:49.119416 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:49.119420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:49.119422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:49.119856 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:49.120099 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:49.120425 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 19:31:49.120674 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":343,"threads_started":1}
I20260810 19:31:49.122622 31056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:49.209769 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 30908
I20260810 19:31:49.229367 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 31069
Aug 10 19:31:49 dist-test-slave-tm1g krb5kdc[30637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:31:49.246810 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:49.230523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.29.233.131:34437)
0810 19:31:49.231188 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:49.231196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:49.231226 (+ 30us) server_negotiation.cc:407] Connection header received
0810 19:31:49.231288 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:49.231295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:49.231530 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:49.231707 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:49.233080 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:49.234117 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:49.235549 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:49.236139 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:49.236291 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:49.240577 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:49.240588 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:49.240604 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:49.240638 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:49.243855 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:49.244604 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:49.244611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:49.244613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:49.244748 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:49.245129 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:49.245133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:49.245136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:49.245499 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:49.245844 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:49.246143 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 19:31:49.246463 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":349,"threads_started":1}
I20260810 19:31:49.248150 31222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:49.334257 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 31069
I20260810 19:31:49.356002 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 30652
I20260810 19:31:49.357347 30724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:49.463099 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 30652
2026-08-10T19:31:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4305 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:31:49 dist-test-slave-tm1g krb5kdc[31255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:31:49 dist-test-slave-tm1g krb5kdc[31255](info): set up 2 sockets
Aug 10 19:31:49 dist-test-slave-tm1g krb5kdc[31255](info): commencing operation
krb5kdc: starting...
W20260810 19:31:50.619884 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:31:50 dist-test-slave-tm1g krb5kdc[31255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:31:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:31:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:31:50.842641 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43031
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:43031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:51.184537 31271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:51.184917 31271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:51.185070 31271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:51.192435 31271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:31:51.192595 31271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:51.192700 31271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:51.192804 31271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:31:51.192878 31271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:31:51.203473 31271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:43031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:51.205152 31271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:51.206768 31271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:51.220062 31277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:51.220011 31278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:51.219962 31280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:51.224080 31271 server_base.cc:1034] running on GCE node
I20260810 19:31:51.225028 31271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:51.226706 31271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:51.227928 31271 hybrid_clock.cc:648] HybridClock initialized: now 1786390311227878 us; error 38 us; skew 500 ppm
I20260810 19:31:51.228318 31271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:51 dist-test-slave-tm1g krb5kdc[31255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390311, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:51.231778 31271 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:31:51.233649 31271 webserver.cc:460] Webserver started at http://127.29.233.190:37323/ using document root <none> and password file <none>
I20260810 19:31:51.234499 31271 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:51.234628 31271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:51.234983 31271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:51.238102 31271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "e3de5ba1de3544d68a6e9c163b95a760"
format_stamp: "Formatted at 2026-08-10 19:31:51 on dist-test-slave-tm1g"
server_key: "5c8ff754364b8fdbe9c22b52dfb6a3f9"
server_key_iv: "846036d72612066991970fa764c2203c"
server_key_version: "encryptionkey@0"
I20260810 19:31:51.239065 31271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "e3de5ba1de3544d68a6e9c163b95a760"
format_stamp: "Formatted at 2026-08-10 19:31:51 on dist-test-slave-tm1g"
server_key: "5c8ff754364b8fdbe9c22b52dfb6a3f9"
server_key_iv: "846036d72612066991970fa764c2203c"
server_key_version: "encryptionkey@0"
I20260810 19:31:51.244583 31271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:31:51.248147 31289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:51.249575 31271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:31:51.249786 31271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "e3de5ba1de3544d68a6e9c163b95a760"
format_stamp: "Formatted at 2026-08-10 19:31:51 on dist-test-slave-tm1g"
server_key: "5c8ff754364b8fdbe9c22b52dfb6a3f9"
server_key_iv: "846036d72612066991970fa764c2203c"
server_key_version: "encryptionkey@0"
I20260810 19:31:51.250006 31271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:51.277812 31271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:51.282835 31271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:51.283289 31271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:51.303731 31271 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:43031
I20260810 19:31:51.303771 31346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:43031 every 8 connection(s)
I20260810 19:31:51.305225 31271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:31:51.309670 31347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:51.314826 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 31271
I20260810 19:31:51.315117 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:31:51.316042 30630 external_mini_cluster.cc:1428] Setting key 76a5dd7e1c61a5f1c3e80178f59c89d3
I20260810 19:31:51.322515 31347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760: Bootstrap starting.
I20260810 19:31:51.326560 31347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:51.327818 31347 log.cc:824] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:51.331193 31347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760: No bootstrap required, opened a new log
Aug 10 19:31:51 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:51.341867 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:51.318387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.29.233.190:43031)
0810 19:31:51.319327 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:51.319341 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:51.319386 (+ 45us) server_negotiation.cc:407] Connection header received
0810 19:31:51.321225 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:51.321252 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:51.321691 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:51.322046 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:51.324064 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:51.325498 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:51.327381 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:51.328161 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:51.328338 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:51.333536 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:51.333547 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:51.333563 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:51.333613 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:51.337059 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:51.338035 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:51.338043 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:51.338049 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:51.338241 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:51.338780 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:51.338784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:51.338786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:51.339390 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:51.339637 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:51.340301 (+ 664us) server_negotiation.cc:299] Negotiation successful
0810 19:31:51.340771 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":473,"threads_started":1}
I20260810 19:31:51.341836 31347 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3de5ba1de3544d68a6e9c163b95a760" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43031 } }
I20260810 19:31:51.342358 31347 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:51.342509 31347 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3de5ba1de3544d68a6e9c163b95a760, State: Initialized, Role: FOLLOWER
I20260810 19:31:51.343122 31347 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3de5ba1de3544d68a6e9c163b95a760" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43031 } }
I20260810 19:31:51.343398 31347 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:31:51.343561 31347 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:31:51.343770 31347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:51.346349 31347 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3de5ba1de3544d68a6e9c163b95a760" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43031 } }
I20260810 19:31:51.346907 31347 leader_election.cc:302] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3de5ba1de3544d68a6e9c163b95a760; no voters:
I20260810 19:31:51.347608 31347 leader_election.cc:288] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:31:51.347766 31352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:31:51.349030 31352 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [term 1 LEADER]: Becoming Leader. State: Replica: e3de5ba1de3544d68a6e9c163b95a760, State: Running, Role: LEADER
I20260810 19:31:51.349586 31352 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3de5ba1de3544d68a6e9c163b95a760" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43031 } }
I20260810 19:31:51.349893 31347 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:31:51.352632 31353 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3de5ba1de3544d68a6e9c163b95a760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3de5ba1de3544d68a6e9c163b95a760" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43031 } } }
I20260810 19:31:51.353085 31353 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [sys.catalog]: This master's current role is: LEADER
I20260810 19:31:51.352602 31354 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3de5ba1de3544d68a6e9c163b95a760. Latest consensus state: current_term: 1 leader_uuid: "e3de5ba1de3544d68a6e9c163b95a760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3de5ba1de3544d68a6e9c163b95a760" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43031 } } }
I20260810 19:31:51.353614 31354 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760 [sys.catalog]: This master's current role is: LEADER
I20260810 19:31:51.359565 31362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:31:51.364238 31362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:31:51.373221 31362 catalog_manager.cc:1347] Generated new cluster ID: 963f930043f0411ebe4ab855ae8fd2a7
I20260810 19:31:51.373363 31362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:31:51.391045 31362 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:31:51.392771 31362 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:31:51.406574 31362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3de5ba1de3544d68a6e9c163b95a760: Generated new TSK 0
I20260810 19:31:51.407537 31362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:31:51.466930 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43031
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:51.827668 31376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:51.828035 31376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:51.828197 31376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:51.836118 31376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:51.836292 31376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:51.836472 31376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:31:51.846815 31376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:51.848253 31376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:51.849731 31376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:51.864507 31384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:51.864564 31382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:51.864540 31386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:51.865195 31376 server_base.cc:1034] running on GCE node
I20260810 19:31:51.865922 31376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:51.867023 31376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:51.868300 31376 hybrid_clock.cc:648] HybridClock initialized: now 1786390311868237 us; error 49 us; skew 500 ppm
I20260810 19:31:51.868680 31376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:51 dist-test-slave-tm1g krb5kdc[31255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390311, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:51.872093 31376 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:31:51.873692 31376 webserver.cc:460] Webserver started at http://127.29.233.129:42547/ using document root <none> and password file <none>
I20260810 19:31:51.874485 31376 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:51.874629 31376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:51.874976 31376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:51.877987 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "de8687fc32d04c389cfed0c829d12dc1"
format_stamp: "Formatted at 2026-08-10 19:31:51 on dist-test-slave-tm1g"
server_key: "08a88cbec834799c20468cd3713decbd"
server_key_iv: "df20a4b2a914e3b27e71d443ea931c7a"
server_key_version: "encryptionkey@0"
I20260810 19:31:51.878888 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "de8687fc32d04c389cfed0c829d12dc1"
format_stamp: "Formatted at 2026-08-10 19:31:51 on dist-test-slave-tm1g"
server_key: "08a88cbec834799c20468cd3713decbd"
server_key_iv: "df20a4b2a914e3b27e71d443ea931c7a"
server_key_version: "encryptionkey@0"
I20260810 19:31:51.884300 31376 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:31:51.888159 31396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:51.889637 31376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:31:51.889843 31376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "de8687fc32d04c389cfed0c829d12dc1"
format_stamp: "Formatted at 2026-08-10 19:31:51 on dist-test-slave-tm1g"
server_key: "08a88cbec834799c20468cd3713decbd"
server_key_iv: "df20a4b2a914e3b27e71d443ea931c7a"
server_key_version: "encryptionkey@0"
I20260810 19:31:51.890076 31376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:51.907729 31376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:51.912585 31376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:51.912904 31376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:51.913918 31376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:31:51.915436 31376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:51.915549 31376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:51.915696 31376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:51.915779 31376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:51.951747 31376 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:37727
I20260810 19:31:51.951812 31523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:37727 every 8 connection(s)
I20260810 19:31:51.953166 31376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:31:51.962111 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 31376
I20260810 19:31:51.962378 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:31:51.963251 30630 external_mini_cluster.cc:1428] Setting key 2282a694e21e53b60a6ca6f95b17c697
Aug 10 19:31:51 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390311, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:51.975260 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:51.955943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:49765 (local address 127.29.233.190:43031)
0810 19:31:51.956452 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:51.956459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:51.957465 (+ 1006us) server_negotiation.cc:407] Connection header received
0810 19:31:51.959490 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:51.959494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:51.959570 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:51.959672 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:51.961269 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:51.962450 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:51.964888 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:51.965359 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:51.965511 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:51.969151 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:51.969157 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:51.969160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:51.969194 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:51.971986 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:51.972787 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:51.972794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:51.972798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:51.972933 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:51.973379 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:51.973385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:51.973388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:51.973680 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:51.973841 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:51.974643 (+ 802us) server_negotiation.cc:299] Negotiation successful
0810 19:31:51.974885 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":219,"threads_started":1}
I20260810 19:31:51.976328 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:51.956585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:43031 (local address 127.29.233.129:49765)
0810 19:31:51.957305 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:51.957346 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:51.959188 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:51.959930 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:51.959953 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:51.960502 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:51.961088 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:51.961106 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:51.962684 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:51.962691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:51.964703 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:51.964717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:51.965628 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:51.965636 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:51.965874 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:51.966639 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:51.966667 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:51.968896 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:51.972166 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:51.972176 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:51.972194 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:51.972654 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:51.973047 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:51.973052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:51.973055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:51.973262 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:51.973814 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:51.973822 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:51.974197 (+ 375us) client_negotiation.cc:769] Sending connection context
0810 19:31:51.974532 (+ 335us) client_negotiation.cc:240] Negotiation successful
0810 19:31:51.975135 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":280,"threads_started":1}
I20260810 19:31:51.978082 31524 heartbeater.cc:347] Connected to a master server at 127.29.233.190:43031
I20260810 19:31:51.978463 31524 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:51.979230 31524 heartbeater.cc:511] Master 127.29.233.190:43031 requested a full tablet report, sending...
I20260810 19:31:51.981745 31310 ts_manager.cc:194] Registered new tserver with Master: de8687fc32d04c389cfed0c829d12dc1 (127.29.233.129:37727)
I20260810 19:31:51.984097 31310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:49765
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:31:52.017354 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43031
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:52.438210 31530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:52.438602 31530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:52.438709 31530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:52.446241 31530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:52.446452 31530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:52.446590 31530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:31:52.458122 31530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:52.459599 31530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:52.461505 31530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:52.478330 31535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:52.478703 31538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:52.478789 31536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:52.480384 31530 server_base.cc:1034] running on GCE node
I20260810 19:31:52.481077 31530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:52.482149 31530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:52.483389 31530 hybrid_clock.cc:648] HybridClock initialized: now 1786390312483336 us; error 43 us; skew 500 ppm
I20260810 19:31:52.483783 31530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:52 dist-test-slave-tm1g krb5kdc[31255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390312, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:52.496542 31530 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:31:52.498432 31530 webserver.cc:460] Webserver started at http://127.29.233.130:34505/ using document root <none> and password file <none>
I20260810 19:31:52.499258 31530 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:52.499423 31530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:52.499773 31530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:52.503093 31530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "f85fdd74d4a24777af44bbb5de0aea3e"
format_stamp: "Formatted at 2026-08-10 19:31:52 on dist-test-slave-tm1g"
server_key: "e57eaf3681ff467a17c0c9d7801972a4"
server_key_iv: "86ff7b9af565ed33bad8d96e2b7c0186"
server_key_version: "encryptionkey@0"
I20260810 19:31:52.503961 31530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "f85fdd74d4a24777af44bbb5de0aea3e"
format_stamp: "Formatted at 2026-08-10 19:31:52 on dist-test-slave-tm1g"
server_key: "e57eaf3681ff467a17c0c9d7801972a4"
server_key_iv: "86ff7b9af565ed33bad8d96e2b7c0186"
server_key_version: "encryptionkey@0"
I20260810 19:31:52.509711 31530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 19:31:52.513751 31548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:52.515426 31530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:31:52.515640 31530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "f85fdd74d4a24777af44bbb5de0aea3e"
format_stamp: "Formatted at 2026-08-10 19:31:52 on dist-test-slave-tm1g"
server_key: "e57eaf3681ff467a17c0c9d7801972a4"
server_key_iv: "86ff7b9af565ed33bad8d96e2b7c0186"
server_key_version: "encryptionkey@0"
I20260810 19:31:52.515868 31530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:52.540028 31530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:52.544888 31530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:52.545233 31530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:52.546355 31530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:31:52.547914 31530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:52.548015 31530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:52.548164 31530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:52.548234 31530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:52.584618 31530 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:40125
I20260810 19:31:52.584669 31677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:40125 every 8 connection(s)
I20260810 19:31:52.586031 31530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:31:52.588516 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 31530
I20260810 19:31:52.588737 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:31:52.589346 30630 external_mini_cluster.cc:1428] Setting key cf54851cabd56c503deae3fdaa33588e
Aug 10 19:31:52 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390312, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
I20260810 19:31:52.611902 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:52.588891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:44965 (local address 127.29.233.190:43031)
0810 19:31:52.589353 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:52.589358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:52.590718 (+ 1360us) server_negotiation.cc:407] Connection header received
0810 19:31:52.593028 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:52.593036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:52.593138 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:52.593272 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:52.595102 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:52.596306 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:52.598028 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:52.598803 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:52.598961 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:52.603847 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:52.603852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:52.603855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:52.603901 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:52.606949 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:52.608410 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:52.608416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:52.608419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:52.608760 (+ 341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:52.609422 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:52.609427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:52.609430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:52.609971 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:52.610355 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:52.611210 (+ 855us) server_negotiation.cc:299] Negotiation successful
0810 19:31:52.611491 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260810 19:31:52.613153 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:52.589534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:43031 (local address 127.29.233.130:44965)
0810 19:31:52.590547 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:52.590589 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:52.592709 (+ 2120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:52.593540 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:52.593550 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:52.594155 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:52.594880 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:52.594901 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:52.596479 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:52.596484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:52.597861 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:52.597881 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:52.599031 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:52.599036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:52.599258 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:52.600142 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:52.600179 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:52.603540 (+ 3361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:52.607137 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:52.607147 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:52.607168 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:52.608148 (+ 980us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:52.608949 (+ 801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:52.608955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:52.608961 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:52.609260 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:52.610252 (+ 992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:52.610263 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:52.610753 (+ 490us) client_negotiation.cc:769] Sending connection context
0810 19:31:52.611074 (+ 321us) client_negotiation.cc:240] Negotiation successful
0810 19:31:52.611702 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":409,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
I20260810 19:31:52.614843 31678 heartbeater.cc:347] Connected to a master server at 127.29.233.190:43031
I20260810 19:31:52.615187 31678 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:52.616082 31678 heartbeater.cc:511] Master 127.29.233.190:43031 requested a full tablet report, sending...
I20260810 19:31:52.617942 31310 ts_manager.cc:194] Registered new tserver with Master: f85fdd74d4a24777af44bbb5de0aea3e (127.29.233.130:40125)
I20260810 19:31:52.619041 31310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:44965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:31:52.642989 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43031
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:31:52.987669 31524 heartbeater.cc:503] Master 127.29.233.190:43031 was elected leader, sending a full tablet report...
W20260810 19:31:52.999662 31685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:53.000017 31685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:53.000195 31685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:53.007814 31685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:53.007977 31685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:53.008148 31685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:31:53.018638 31685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:53.020140 31685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:53.021616 31685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:53.036454 31693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:53.036518 31695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:53.036432 31690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:53.037367 31685 server_base.cc:1034] running on GCE node
I20260810 19:31:53.038021 31685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:53.039139 31685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:53.040357 31685 hybrid_clock.cc:648] HybridClock initialized: now 1786390313040304 us; error 43 us; skew 500 ppm
I20260810 19:31:53.040705 31685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:53.044091 31685 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:31:53.045773 31685 webserver.cc:460] Webserver started at http://127.29.233.131:38533/ using document root <none> and password file <none>
I20260810 19:31:53.046607 31685 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:53.046756 31685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:53.047088 31685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:53.050238 31685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "00d28b8a7a7a430bbf50aee13a798f2b"
format_stamp: "Formatted at 2026-08-10 19:31:53 on dist-test-slave-tm1g"
server_key: "309206b001ea9cffbf3acbc1601800c3"
server_key_iv: "dc043255d191f44b3213d0c7a310bfc8"
server_key_version: "encryptionkey@0"
I20260810 19:31:53.051168 31685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "00d28b8a7a7a430bbf50aee13a798f2b"
format_stamp: "Formatted at 2026-08-10 19:31:53 on dist-test-slave-tm1g"
server_key: "309206b001ea9cffbf3acbc1601800c3"
server_key_iv: "dc043255d191f44b3213d0c7a310bfc8"
server_key_version: "encryptionkey@0"
I20260810 19:31:53.056986 31685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 19:31:53.060716 31703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:53.062466 31685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 19:31:53.062651 31685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "00d28b8a7a7a430bbf50aee13a798f2b"
format_stamp: "Formatted at 2026-08-10 19:31:53 on dist-test-slave-tm1g"
server_key: "309206b001ea9cffbf3acbc1601800c3"
server_key_iv: "dc043255d191f44b3213d0c7a310bfc8"
server_key_version: "encryptionkey@0"
I20260810 19:31:53.062872 31685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:53.086519 31685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:53.091871 31685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:53.092219 31685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:53.093468 31685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:31:53.095126 31685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:53.095212 31685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:53.095381 31685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:53.095453 31685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:53.131259 31685 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:40679
I20260810 19:31:53.131352 31827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:40679 every 8 connection(s)
I20260810 19:31:53.132645 31685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:31:53.137801 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 31685
I20260810 19:31:53.138019 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:31:53.138512 30630 external_mini_cluster.cc:1428] Setting key 1ab82c9a2bc0b6d59510e1eb4a322ae9
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:53.154838 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.135485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:41921 (local address 127.29.233.190:43031)
0810 19:31:53.135912 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.135916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.136961 (+ 1045us) server_negotiation.cc:407] Connection header received
0810 19:31:53.139052 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.139057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.139142 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.139238 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.141223 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.142213 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.143890 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.144299 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.144416 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.148511 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.148517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.148520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.148552 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.151289 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.152106 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.152112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.152114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.152261 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.152767 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.152771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.152774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.153142 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.153296 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.154291 (+ 995us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.154561 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":206,"threads_started":1}
I20260810 19:31:53.155987 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.136028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:43031 (local address 127.29.233.131:41921)
0810 19:31:53.136789 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:53.136829 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:53.138771 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:53.139486 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:53.139496 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:53.140204 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:53.141006 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.141075 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.142454 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.142459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.143736 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.143749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.144543 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.144549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.144766 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.145507 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:53.145531 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:53.148287 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:53.151452 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:53.151480 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:53.151499 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:53.151969 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:53.152394 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:53.152398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:53.152401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:53.152653 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:53.153264 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:53.153277 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:53.153757 (+ 480us) client_negotiation.cc:769] Sending connection context
0810 19:31:53.154139 (+ 382us) client_negotiation.cc:240] Negotiation successful
0810 19:31:53.154748 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":297,"threads_started":1}
I20260810 19:31:53.157565 31828 heartbeater.cc:347] Connected to a master server at 127.29.233.190:43031
I20260810 19:31:53.157922 31828 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:53.158739 31828 heartbeater.cc:511] Master 127.29.233.190:43031 requested a full tablet report, sending...
I20260810 19:31:53.160241 31310 ts_manager.cc:194] Registered new tserver with Master: 00d28b8a7a7a430bbf50aee13a798f2b (127.29.233.131:40679)
I20260810 19:31:53.161118 31310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:41921
I20260810 19:31:53.167521 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:31:53.187242 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.174256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.29.233.190:43031)
0810 19:31:53.174462 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.174466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.174970 (+ 504us) server_negotiation.cc:407] Connection header received
0810 19:31:53.175120 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.175124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.175193 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.175277 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.176761 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.177510 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.178900 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.179294 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.179415 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.181820 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.181824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.181828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.181859 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.184554 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.185349 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.185354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.185356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.185492 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.185844 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.185848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.185850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.186204 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.186486 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.186745 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.186913 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 19:31:53.194162 31310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:31:53.201699 31310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:31:53.230928 31847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.215155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:37727 (local address 127.0.0.1:42266)
0810 19:31:53.219572 (+ 4417us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:53.219625 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:53.219888 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:53.221728 (+ 1840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:53.221737 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:53.221765 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:53.222236 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.222250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.225065 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.225104 (+ 39us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.227712 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.227731 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.229829 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.229835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.230062 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.230123 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 19:31:53.230259 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 19:31:53.230543 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4045,"thread_start_us":6082,"threads_started":1}
I20260810 19:31:53.232643 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.220258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:40125 (local address 127.0.0.1:40070)
0810 19:31:53.222886 (+ 2628us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:53.222904 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:53.223011 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:53.224105 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:53.224112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:53.224142 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:53.224724 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.224742 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.226915 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.226931 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.229478 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.229496 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.231507 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.231514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.231752 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.231907 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 19:31:53.231992 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 19:31:53.232140 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2504,"mutex_wait_us":1,"thread_start_us":3175,"threads_started":1}
I20260810 19:31:53.232749 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.216083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.29.233.131:40679)
0810 19:31:53.216904 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.216917 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.221768 (+ 4851us) server_negotiation.cc:407] Connection header received
0810 19:31:53.222738 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.222751 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.223052 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.223263 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:53.224821 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.226766 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.229155 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.231120 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.231319 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.231513 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.232209 (+ 696us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.232404 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":430,"threads_started":1}
I20260810 19:31:53.232877 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.215376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.29.233.129:37727)
0810 19:31:53.220967 (+ 5591us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.220980 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.221021 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:31:53.221154 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.221166 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.221472 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.222509 (+ 1037us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:53.222762 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.224537 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.227909 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.229601 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.231903 (+ 2302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.232111 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.232189 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.232421 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5354,"thread_start_us":6909,"threads_started":1}
I20260810 19:31:53.233569 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.215620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:40679 (local address 127.0.0.1:44490)
0810 19:31:53.221604 (+ 5984us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:53.221630 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:53.222347 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:53.223624 (+ 1277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:53.223627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:53.223647 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:53.224387 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.224407 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.226941 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.226945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.228971 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.228987 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.231687 (+ 2700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.231693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.231975 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.232006 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 19:31:53.233043 (+ 1037us) client_negotiation.cc:240] Negotiation successful
0810 19:31:53.233197 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5818,"mutex_wait_us":142,"thread_start_us":3369,"threads_started":1}
I20260810 19:31:53.233603 31850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.220484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.29.233.130:40125)
0810 19:31:53.223519 (+ 3035us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.223528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.223553 (+ 25us) server_negotiation.cc:407] Connection header received
0810 19:31:53.223640 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.223647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.223864 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.224053 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:53.225035 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.226706 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.229697 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.231241 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.232425 (+ 1184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.232603 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.232676 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.233277 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2871,"thread_start_us":2741,"threads_started":1}
I20260810 19:31:53.237246 31756 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1c055426df4caca90a91c29aad1162 (DEFAULT_TABLE table=test-table [id=659db6b4e5db405fbdcf4da8ef175899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:53.237160 31451 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1c055426df4caca90a91c29aad1162 (DEFAULT_TABLE table=test-table [id=659db6b4e5db405fbdcf4da8ef175899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:53.237160 31604 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1c055426df4caca90a91c29aad1162 (DEFAULT_TABLE table=test-table [id=659db6b4e5db405fbdcf4da8ef175899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:53.238718 31451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c1c055426df4caca90a91c29aad1162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:53.238736 31756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c1c055426df4caca90a91c29aad1162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:53.238739 31604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c1c055426df4caca90a91c29aad1162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:53.252141 31851 tablet_bootstrap.cc:492] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b: Bootstrap starting.
I20260810 19:31:53.255220 31852 tablet_bootstrap.cc:492] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e: Bootstrap starting.
I20260810 19:31:53.255728 31853 tablet_bootstrap.cc:492] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1: Bootstrap starting.
I20260810 19:31:53.258595 31852 tablet_bootstrap.cc:654] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:53.258581 31851 tablet_bootstrap.cc:654] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:53.259443 31853 tablet_bootstrap.cc:654] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:53.259739 31852 log.cc:824] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:53.259936 31851 log.cc:824] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:53.260703 31853 log.cc:824] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:53.263046 31852 tablet_bootstrap.cc:492] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e: No bootstrap required, opened a new log
I20260810 19:31:53.263345 31852 ts_tablet_manager.cc:1397] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260810 19:31:53.263643 31851 tablet_bootstrap.cc:492] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b: No bootstrap required, opened a new log
I20260810 19:31:53.263650 31853 tablet_bootstrap.cc:492] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1: No bootstrap required, opened a new log
I20260810 19:31:53.263916 31851 ts_tablet_manager.cc:1397] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260810 19:31:53.264619 31853 ts_tablet_manager.cc:1397] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 19:31:53.272173 31852 raft_consensus.cc:348] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.272630 31852 raft_consensus.cc:374] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:53.273038 31852 raft_consensus.cc:729] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85fdd74d4a24777af44bbb5de0aea3e, State: Initialized, Role: FOLLOWER
I20260810 19:31:53.273902 31852 consensus_queue.cc:260] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.275270 31851 raft_consensus.cc:348] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.275765 31851 raft_consensus.cc:374] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:53.275873 31851 raft_consensus.cc:729] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d28b8a7a7a430bbf50aee13a798f2b, State: Initialized, Role: FOLLOWER
I20260810 19:31:53.276465 31678 heartbeater.cc:503] Master 127.29.233.190:43031 was elected leader, sending a full tablet report...
I20260810 19:31:53.276650 31851 consensus_queue.cc:260] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.276612 31853 raft_consensus.cc:348] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.277040 31853 raft_consensus.cc:374] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:53.277051 31852 ts_tablet_manager.cc:1428] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260810 19:31:53.277148 31853 raft_consensus.cc:729] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8687fc32d04c389cfed0c829d12dc1, State: Initialized, Role: FOLLOWER
I20260810 19:31:53.277993 31853 consensus_queue.cc:260] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.280285 31853 ts_tablet_manager.cc:1428] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260810 19:31:53.281284 31828 heartbeater.cc:503] Master 127.29.233.190:43031 was elected leader, sending a full tablet report...
I20260810 19:31:53.282125 31851 ts_tablet_manager.cc:1428] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b: Time spent starting tablet: real 0.018s user 0.014s sys 0.003s
W20260810 19:31:53.338279 31680 tablet.cc:2369] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:31:53.350507 31857 raft_consensus.cc:482] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:31:53.350863 31857 raft_consensus.cc:504] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.352907 31857 leader_election.cc:288] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de8687fc32d04c389cfed0c829d12dc1 (127.29.233.129:37727), 00d28b8a7a7a430bbf50aee13a798f2b (127.29.233.131:40679)
I20260810 19:31:53.361243 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.353712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:49187 (local address 127.29.233.131:40679)
0810 19:31:53.354185 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.354191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.354390 (+ 199us) server_negotiation.cc:407] Connection header received
0810 19:31:53.354672 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.354678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.354784 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.354883 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:53.355661 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.356958 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.359280 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.360099 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.360291 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.360387 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.360815 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.360964 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260810 19:31:53.361347 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.353481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:33613 (local address 127.29.233.129:37727)
0810 19:31:53.354006 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.354015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.354823 (+ 808us) server_negotiation.cc:407] Connection header received
0810 19:31:53.355079 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.355085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.355173 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.355282 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:53.355984 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.356750 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.359304 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.360124 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.360315 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.360398 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.360806 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.360947 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367}
I20260810 19:31:53.361683 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.353366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:37727 (local address 127.29.233.130:33613)
0810 19:31:53.354679 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:53.354697 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:53.354843 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:53.355302 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:53.355307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:53.355331 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:53.355782 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.355805 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.356901 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.356908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.359009 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.359022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.360410 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.360418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.360632 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.360658 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 19:31:53.360775 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 19:31:53.360943 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"mutex_wait_us":31,"thread_start_us":861,"threads_started":1}
I20260810 19:31:53.361898 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.353167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:40679 (local address 127.29.233.130:49187)
0810 19:31:53.354245 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:53.354270 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:53.354492 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:53.354941 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:53.354946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:53.355009 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:53.355495 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.355530 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.357102 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.357106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.358995 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:53.359009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.360364 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.360370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:53.360639 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.360663 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:31:53.360773 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 19:31:53.360946 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":113,"thread_start_us":614,"threads_started":1}
I20260810 19:31:53.363031 31781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1c055426df4caca90a91c29aad1162" candidate_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" is_pre_election: true
I20260810 19:31:53.363031 31472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1c055426df4caca90a91c29aad1162" candidate_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8687fc32d04c389cfed0c829d12dc1" is_pre_election: true
I20260810 19:31:53.363670 31781 raft_consensus.cc:2454] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f85fdd74d4a24777af44bbb5de0aea3e in term 0.
I20260810 19:31:53.363670 31472 raft_consensus.cc:2454] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f85fdd74d4a24777af44bbb5de0aea3e in term 0.
I20260810 19:31:53.364426 31549 leader_election.cc:302] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d28b8a7a7a430bbf50aee13a798f2b, f85fdd74d4a24777af44bbb5de0aea3e; no voters:
I20260810 19:31:53.364804 31857 raft_consensus.cc:2781] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:31:53.364974 31857 raft_consensus.cc:482] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:31:53.365093 31857 raft_consensus.cc:3037] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:53.367515 31857 raft_consensus.cc:504] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.368196 31857 leader_election.cc:288] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [CANDIDATE]: Term 1 election: Requested vote from peers de8687fc32d04c389cfed0c829d12dc1 (127.29.233.129:37727), 00d28b8a7a7a430bbf50aee13a798f2b (127.29.233.131:40679)
I20260810 19:31:53.368677 31472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1c055426df4caca90a91c29aad1162" candidate_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de8687fc32d04c389cfed0c829d12dc1"
I20260810 19:31:53.368775 31781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1c055426df4caca90a91c29aad1162" candidate_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d28b8a7a7a430bbf50aee13a798f2b"
I20260810 19:31:53.368919 31472 raft_consensus.cc:3037] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:53.369035 31781 raft_consensus.cc:3037] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:53.371519 31472 raft_consensus.cc:2454] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f85fdd74d4a24777af44bbb5de0aea3e in term 1.
I20260810 19:31:53.371944 31781 raft_consensus.cc:2454] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f85fdd74d4a24777af44bbb5de0aea3e in term 1.
I20260810 19:31:53.372191 31550 leader_election.cc:302] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de8687fc32d04c389cfed0c829d12dc1, f85fdd74d4a24777af44bbb5de0aea3e; no voters:
I20260810 19:31:53.372598 31857 raft_consensus.cc:2781] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:31:53.373076 31857 raft_consensus.cc:686] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [term 1 LEADER]: Becoming Leader. State: Replica: f85fdd74d4a24777af44bbb5de0aea3e, State: Running, Role: LEADER
I20260810 19:31:53.373593 31857 consensus_queue.cc:237] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } }
I20260810 19:31:53.378289 31306 catalog_manager.cc:5495] T 3c1c055426df4caca90a91c29aad1162 P f85fdd74d4a24777af44bbb5de0aea3e reported cstate change: term changed from 0 to 1, leader changed from <none> to f85fdd74d4a24777af44bbb5de0aea3e (127.29.233.130). New cstate: current_term: 1 leader_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de8687fc32d04c389cfed0c829d12dc1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 37727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d28b8a7a7a430bbf50aee13a798f2b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f85fdd74d4a24777af44bbb5de0aea3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40125 } health_report { overall_health: HEALTHY } } }
W20260810 19:31:53.384408 31829 tablet.cc:2369] T 3c1c055426df4caca90a91c29aad1162 P 00d28b8a7a7a430bbf50aee13a798f2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:31:53.436911 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.421208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.29.233.129:37727)
0810 19:31:53.421427 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.421553 (+ 126us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.422055 (+ 502us) server_negotiation.cc:407] Connection header received
0810 19:31:53.422285 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.422289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.422434 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.422548 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.424142 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.424949 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.426240 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.426734 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.426882 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.431070 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.431085 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.431101 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.431136 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.433963 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.434761 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.434769 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.434771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.434920 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.435293 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.435299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.435302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.435678 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.436003 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.436286 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.436528 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 19:31:53.455868 31525 tablet.cc:2369] T 3c1c055426df4caca90a91c29aad1162 P de8687fc32d04c389cfed0c829d12dc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:31:53.492941 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.477615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.29.233.129:37727)
0810 19:31:53.477911 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.477918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.478125 (+ 207us) server_negotiation.cc:407] Connection header received
0810 19:31:53.478278 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.478282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.478407 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.478501 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.479951 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.480779 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.482105 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.482644 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.482745 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.487144 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.487151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.487156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.487202 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.490094 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.490925 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.490930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.490933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.491087 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.491547 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.491551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.491553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.491831 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.492052 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.492341 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.492533 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:31:53.551043 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.534762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.29.233.129:37727)
0810 19:31:53.535056 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.535064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.535409 (+ 345us) server_negotiation.cc:407] Connection header received
0810 19:31:53.535642 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.535648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.535722 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.535845 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.537299 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.538158 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.539818 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.540426 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.540575 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.545049 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.545055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.545059 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.545101 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.548076 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.548857 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.548862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.548865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.549004 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.549405 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.549410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.549412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.549725 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.550044 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.550372 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.550653 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260810 19:31:53.553001 31428 tablet_service.cc:2908] Rejecting scan request for tablet 3c1c055426df4caca90a91c29aad1162: Uninitialized: clean time has not yet been initialized
I20260810 19:31:53.556852 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 31376
I20260810 19:31:53.570838 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.557739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.29.233.129:37727)
0810 19:31:53.557977 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.557987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.558351 (+ 364us) server_negotiation.cc:407] Connection header received
0810 19:31:53.558558 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.558562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.558638 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.558728 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.560014 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.560728 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.561986 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.562507 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.562664 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.565113 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.565118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.565122 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.565157 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.568037 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.568727 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.568734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.568738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.568873 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.569275 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.569281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.569284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.569595 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.569912 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.570227 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.570488 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:31:53.571563 31519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:53.674136 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 31376
I20260810 19:31:53.695201 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 31530
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:31:53.710907 31850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.696090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40084 (local address 127.29.233.130:40125)
0810 19:31:53.696279 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.696284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.696304 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:31:53.696435 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.696438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.696502 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.696579 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.697837 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.698662 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.700194 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.700633 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.700739 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.704969 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.704985 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.705003 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.705046 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.708126 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.708920 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.708928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.708930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.709102 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.709449 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.709452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.709454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.709797 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.710032 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.710397 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.710640 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 19:31:53.711613 31673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:53.807534 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 31530
W20260810 19:31:53.827862 31707 connection.cc:535] server connection from 127.29.233.130:49187 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 19:31:53.828270 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 31685
Aug 10 19:31:53 dist-test-slave-tm1g krb5kdc[31255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:31:53.844508 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:53.829084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.29.233.131:40679)
0810 19:31:53.829276 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:53.829283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:53.829368 (+ 85us) server_negotiation.cc:407] Connection header received
0810 19:31:53.829594 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:53.829600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:53.829679 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:53.829789 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:53.831223 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.831984 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.833352 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:53.833782 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:53.833927 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:53.838545 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:53.838562 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:53.838580 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:53.838625 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:53.841426 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.842284 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.842291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.842294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.842495 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:53.842903 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:53.842908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:53.842911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:53.843297 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:53.843599 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:53.843915 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 19:31:53.844150 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:31:53.845162 31823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:53.934016 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 31685
I20260810 19:31:53.953202 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 31271
I20260810 19:31:53.954548 31342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:54.067523 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 31271
2026-08-10T19:31:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4606 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:31:54 dist-test-slave-tm1g krb5kdc[31884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:31:54 dist-test-slave-tm1g krb5kdc[31884](info): set up 2 sockets
Aug 10 19:31:54 dist-test-slave-tm1g krb5kdc[31884](info): commencing operation
krb5kdc: starting...
W20260810 19:31:55.231024 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:31:55 dist-test-slave-tm1g krb5kdc[31884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:31:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:31:55.486783 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33175
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:33175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:55.826865 31898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:55.827183 31898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:55.827342 31898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:55.835053 31898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:31:55.835179 31898 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 19:31:55.835273 31898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:55.835382 31898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:55.835489 31898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:31:55.835600 31898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:31:55.845930 31898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:33175
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:55.847517 31898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:55.848896 31898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:55.860989 31906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:55.861348 31903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:55.861172 31904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:55.862787 31898 server_base.cc:1034] running on GCE node
I20260810 19:31:55.863641 31898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:55.865142 31898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:55.866452 31898 hybrid_clock.cc:648] HybridClock initialized: now 1786390315866346 us; error 58 us; skew 500 ppm
I20260810 19:31:55.866832 31898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:55 dist-test-slave-tm1g krb5kdc[31884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390315, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:55.875447 31898 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:31:55.877110 31898 webserver.cc:460] Webserver started at http://127.29.233.190:33079/ using document root <none> and password file <none>
I20260810 19:31:55.877921 31898 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:55.878060 31898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:55.878464 31898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:55.881567 31898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "ae0dcfde099a44a797f33abc6bd44a94"
format_stamp: "Formatted at 2026-08-10 19:31:55 on dist-test-slave-tm1g"
server_key: "55ddf2fa2548157366375c4ca4d5d484"
server_key_iv: "410a9217196fa4140f3c28c2be964810"
server_key_version: "encryptionkey@0"
I20260810 19:31:55.882452 31898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "ae0dcfde099a44a797f33abc6bd44a94"
format_stamp: "Formatted at 2026-08-10 19:31:55 on dist-test-slave-tm1g"
server_key: "55ddf2fa2548157366375c4ca4d5d484"
server_key_iv: "410a9217196fa4140f3c28c2be964810"
server_key_version: "encryptionkey@0"
I20260810 19:31:55.887789 31898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 19:31:55.891671 31913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:55.893641 31898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 19:31:55.893843 31898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "ae0dcfde099a44a797f33abc6bd44a94"
format_stamp: "Formatted at 2026-08-10 19:31:55 on dist-test-slave-tm1g"
server_key: "55ddf2fa2548157366375c4ca4d5d484"
server_key_iv: "410a9217196fa4140f3c28c2be964810"
server_key_version: "encryptionkey@0"
I20260810 19:31:55.894148 31898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:55.937888 31898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:55.942106 31898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:55.942395 31898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:55.964716 31898 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:33175
I20260810 19:31:55.964753 31985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:33175 every 8 connection(s)
I20260810 19:31:55.966135 31898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:31:55.970036 31986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:55.971072 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 31898
I20260810 19:31:55.971300 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:31:55.971953 30630 external_mini_cluster.cc:1428] Setting key 7ff7d8d00f623f594c1d76668efffeae
I20260810 19:31:55.984299 31986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94: Bootstrap starting.
Aug 10 19:31:55 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:55.988197 31986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:55.989451 31986 log.cc:824] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:55.992764 31986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94: No bootstrap required, opened a new log
I20260810 19:31:55.995937 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:55.973932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54190 (local address 127.29.233.190:33175)
0810 19:31:55.974784 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:55.974799 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:55.974840 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:31:55.976463 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:55.976484 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:55.976878 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:55.977196 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:55.979029 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:55.980335 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:55.982135 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:55.983051 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:55.983177 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:55.988027 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:55.988035 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:55.988050 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:55.988090 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:55.991814 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:55.992547 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:55.992554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:55.992560 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:55.992738 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:55.993135 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:55.993140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:55.993143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:55.993675 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:55.993974 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:55.994472 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 19:31:55.994909 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":388,"threads_started":1}
I20260810 19:31:56.001041 31986 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0dcfde099a44a797f33abc6bd44a94" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33175 } }
I20260810 19:31:56.001484 31986 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:56.001601 31986 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0dcfde099a44a797f33abc6bd44a94, State: Initialized, Role: FOLLOWER
I20260810 19:31:56.002228 31986 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0dcfde099a44a797f33abc6bd44a94" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33175 } }
I20260810 19:31:56.002564 31986 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:31:56.002720 31986 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:31:56.002923 31986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:56.005118 31986 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0dcfde099a44a797f33abc6bd44a94" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33175 } }
I20260810 19:31:56.005672 31986 leader_election.cc:302] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae0dcfde099a44a797f33abc6bd44a94; no voters:
I20260810 19:31:56.006363 31986 leader_election.cc:288] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:31:56.006542 31991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:31:56.008225 31991 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [term 1 LEADER]: Becoming Leader. State: Replica: ae0dcfde099a44a797f33abc6bd44a94, State: Running, Role: LEADER
I20260810 19:31:56.008733 31986 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:31:56.008754 31991 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0dcfde099a44a797f33abc6bd44a94" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33175 } }
I20260810 19:31:56.011795 31993 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae0dcfde099a44a797f33abc6bd44a94. Latest consensus state: current_term: 1 leader_uuid: "ae0dcfde099a44a797f33abc6bd44a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0dcfde099a44a797f33abc6bd44a94" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33175 } } }
I20260810 19:31:56.011933 31992 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae0dcfde099a44a797f33abc6bd44a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae0dcfde099a44a797f33abc6bd44a94" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33175 } } }
I20260810 19:31:56.012276 31993 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [sys.catalog]: This master's current role is: LEADER
I20260810 19:31:56.012367 31992 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94 [sys.catalog]: This master's current role is: LEADER
I20260810 19:31:56.014858 32001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:31:56.020471 32001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:31:56.031770 32001 catalog_manager.cc:1347] Generated new cluster ID: b24a50c6cdf24d25bc9235a4eb4cac8b
I20260810 19:31:56.031917 32001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:31:56.053591 32001 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:31:56.055495 32001 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:31:56.061398 32001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94: Generated new TSK 0
I20260810 19:31:56.062239 32001 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:31:56.125579 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33175
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:56.465562 32016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:56.465850 32016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:56.466029 32016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:56.473313 32016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:56.473454 32016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:56.473619 32016 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:31:56.473749 32016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:31:56.483603 32016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:56.484894 32016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:56.486322 32016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:56.500025 32027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:56.500101 32025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:56.500101 32024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:56.500837 32016 server_base.cc:1034] running on GCE node
I20260810 19:31:56.501572 32016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:56.502717 32016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:56.503975 32016 hybrid_clock.cc:648] HybridClock initialized: now 1786390316503906 us; error 42 us; skew 500 ppm
I20260810 19:31:56.504302 32016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:56 dist-test-slave-tm1g krb5kdc[31884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390316, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:56.507666 32016 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:31:56.509153 32016 webserver.cc:460] Webserver started at http://127.29.233.129:43455/ using document root <none> and password file <none>
I20260810 19:31:56.509969 32016 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:56.510105 32016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:56.510541 32016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:56.513864 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "cff8b485a6ca4ddc97916c6b65aaef94"
format_stamp: "Formatted at 2026-08-10 19:31:56 on dist-test-slave-tm1g"
server_key: "961ccdd7aceb7d661cf1554e26ac48de"
server_key_iv: "bc0626aec77b203040840c9d6e7dc20c"
server_key_version: "encryptionkey@0"
I20260810 19:31:56.514768 32016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "cff8b485a6ca4ddc97916c6b65aaef94"
format_stamp: "Formatted at 2026-08-10 19:31:56 on dist-test-slave-tm1g"
server_key: "961ccdd7aceb7d661cf1554e26ac48de"
server_key_iv: "bc0626aec77b203040840c9d6e7dc20c"
server_key_version: "encryptionkey@0"
I20260810 19:31:56.520284 32016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 19:31:56.524142 32036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:56.525596 32016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:31:56.525795 32016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "cff8b485a6ca4ddc97916c6b65aaef94"
format_stamp: "Formatted at 2026-08-10 19:31:56 on dist-test-slave-tm1g"
server_key: "961ccdd7aceb7d661cf1554e26ac48de"
server_key_iv: "bc0626aec77b203040840c9d6e7dc20c"
server_key_version: "encryptionkey@0"
I20260810 19:31:56.526057 32016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:56.544996 32016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:56.549271 32016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:56.549579 32016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:56.552419 32016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:56.552531 32016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:56.552686 32016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:56.552770 32016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:31:56 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390316, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:56.578871 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:56.553699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33175 (local address 127.29.233.129:47143)
0810 19:31:56.554633 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:56.554684 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:56.556709 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:56.557563 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:56.557578 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:56.558400 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:56.559286 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:56.559320 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.561250 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.561258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:56.564129 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:56.564145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.565449 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.565455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:56.565644 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:56.566372 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:56.566417 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:56.569480 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:56.573364 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:56.573376 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:56.573397 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:56.574080 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:56.574539 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:56.574544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:56.574551 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:56.574950 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:56.575727 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:56.575733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:56.576296 (+ 563us) client_negotiation.cc:769] Sending connection context
0810 19:31:56.576617 (+ 321us) client_negotiation.cc:240] Negotiation successful
0810 19:31:56.577262 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":397,"threads_started":1}
I20260810 19:31:56.579433 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:56.553001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:47143 (local address 127.29.233.190:33175)
0810 19:31:56.553725 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:56.553732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:56.554817 (+ 1085us) server_negotiation.cc:407] Connection header received
0810 19:31:56.557047 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:56.557052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:56.557136 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:56.557275 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:56.559539 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.560970 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.564361 (+ 3391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.565159 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.565321 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:56.569756 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:56.569766 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:56.569774 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:56.569824 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:56.573208 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:56.574219 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:56.574225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:56.574229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:56.574420 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:56.575070 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:56.575075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:56.575079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:56.575497 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:56.575649 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:56.578936 (+ 3287us) server_negotiation.cc:299] Negotiation successful
0810 19:31:56.579193 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":467,"threads_started":1}
I20260810 19:31:56.600690 32016 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:46807
I20260810 19:31:56.600752 32172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:46807 every 8 connection(s)
I20260810 19:31:56.601643 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:56.585441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53233 (local address 127.29.233.190:33175)
0810 19:31:56.585608 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:56.585612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:56.586243 (+ 631us) server_negotiation.cc:407] Connection header received
0810 19:31:56.586509 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:56.586518 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:56.586604 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:56.586747 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:56.588575 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.589609 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.591288 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.591687 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.591836 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:56.594519 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:56.594527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:56.594533 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:56.594578 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:56.598363 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:56.599234 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:56.599240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:56.599246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:56.599429 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:56.599937 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:56.599943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:56.599946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:56.600238 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:56.600405 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:56.601004 (+ 599us) server_negotiation.cc:299] Negotiation successful
0810 19:31:56.601231 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 19:31:56.601706 32127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:56.585482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33175 (local address 127.29.233.129:53233)
0810 19:31:56.586137 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:56.586153 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:56.586349 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:56.586965 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:56.586973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:56.587818 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:56.588385 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:56.588405 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.589754 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.589759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:56.591121 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:56.591134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:56.591898 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:56.591906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:56.592071 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:56.592779 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:56.592805 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:56.594259 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:56.598523 (+ 4264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:56.598529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:56.598535 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:56.599092 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:56.599567 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:56.599573 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:56.599579 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:56.599749 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:56.600379 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:56.600384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:56.600632 (+ 248us) client_negotiation.cc:769] Sending connection context
0810 19:31:56.600821 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 19:31:56.601214 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":357,"threads_started":1}
I20260810 19:31:56.602293 32016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:31:56.604543 32173 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33175
I20260810 19:31:56.604782 32173 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:56.605516 32173 heartbeater.cc:511] Master 127.29.233.190:33175 requested a full tablet report, sending...
I20260810 19:31:56.607456 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 32016
I20260810 19:31:56.607512 31933 ts_manager.cc:194] Registered new tserver with Master: cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807)
I20260810 19:31:56.607718 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:31:56.608420 30630 external_mini_cluster.cc:1428] Setting key bc36e7fd86c1574c36db7f640c8662f4
I20260810 19:31:56.609282 31933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:47143
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:31:56.659235 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33175
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:56.992889 32177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:56.993177 32177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:56.993345 32177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:57.000994 32177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:57.001102 32177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:57.001187 32177 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:31:57.001338 32177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:31:57.011417 32177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:57.012801 32177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:57.014150 32177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:57.028509 32184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:57.028395 32187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:57.028460 32185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:57.032006 32177 server_base.cc:1034] running on GCE node
I20260810 19:31:57.032688 32177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:57.033701 32177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:57.034952 32177 hybrid_clock.cc:648] HybridClock initialized: now 1786390317034887 us; error 41 us; skew 500 ppm
I20260810 19:31:57.035296 32177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:57 dist-test-slave-tm1g krb5kdc[31884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:57.038568 32177 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:31:57.040017 32177 webserver.cc:460] Webserver started at http://127.29.233.130:40987/ using document root <none> and password file <none>
I20260810 19:31:57.040786 32177 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:57.040917 32177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:57.041247 32177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:57.044224 32177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "86d93c6c47894b1ca1f8e26adbabc958"
format_stamp: "Formatted at 2026-08-10 19:31:57 on dist-test-slave-tm1g"
server_key: "da66345503b55d67190f7177ed80f556"
server_key_iv: "853a216298fd4f6a30aca0e182c4a1e7"
server_key_version: "encryptionkey@0"
I20260810 19:31:57.045035 32177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "86d93c6c47894b1ca1f8e26adbabc958"
format_stamp: "Formatted at 2026-08-10 19:31:57 on dist-test-slave-tm1g"
server_key: "da66345503b55d67190f7177ed80f556"
server_key_iv: "853a216298fd4f6a30aca0e182c4a1e7"
server_key_version: "encryptionkey@0"
I20260810 19:31:57.050343 32177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260810 19:31:57.053740 32194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:57.055336 32177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 19:31:57.055532 32177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "86d93c6c47894b1ca1f8e26adbabc958"
format_stamp: "Formatted at 2026-08-10 19:31:57 on dist-test-slave-tm1g"
server_key: "da66345503b55d67190f7177ed80f556"
server_key_iv: "853a216298fd4f6a30aca0e182c4a1e7"
server_key_version: "encryptionkey@0"
I20260810 19:31:57.055764 32177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:57.075335 32177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:57.079867 32177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:57.080165 32177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:57.083052 32177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:57.083160 32177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:57.083324 32177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:57.083411 32177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:31:57 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:57.109761 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.083690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:39987 (local address 127.29.233.190:33175)
0810 19:31:57.083892 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.083898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.085436 (+ 1538us) server_negotiation.cc:407] Connection header received
0810 19:31:57.087774 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.087780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.087883 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.088012 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:57.090845 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.091850 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.095187 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.095817 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.095983 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.101256 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:57.101262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:57.101265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:57.101302 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:57.105142 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.106273 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.106279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.106288 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.106527 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.107160 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.107167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.107173 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.107592 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:57.107775 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.108880 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.109207 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:31:57.110885 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.084337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33175 (local address 127.29.233.130:39987)
0810 19:31:57.085219 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.085271 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.087472 (+ 2201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.088462 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.088477 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.089483 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:57.090599 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.090637 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.092060 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.092068 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.094997 (+ 2929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.095017 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.096121 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.096132 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.096389 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.097334 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:57.097357 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:57.100914 (+ 3557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:57.105387 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.105399 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.105422 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.106126 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.106676 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.106683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.106692 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.107003 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.107735 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:57.107745 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:57.108312 (+ 567us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.108732 (+ 420us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.109341 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":398,"threads_started":1}
I20260810 19:31:57.134125 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.116941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:39457 (local address 127.29.233.190:33175)
0810 19:31:57.117182 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.117190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.117898 (+ 708us) server_negotiation.cc:407] Connection header received
0810 19:31:57.118104 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.118108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.118170 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.118251 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:57.120071 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.121407 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.123970 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.124394 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.124557 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.127468 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:57.127473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:57.127476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:57.127509 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:57.130591 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.131669 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.131674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.131677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.131827 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.132295 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.132303 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.132308 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.132838 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:57.133014 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.133543 (+ 529us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.133767 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:31:57.134271 32280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.116926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33175 (local address 127.29.233.130:39457)
0810 19:31:57.117774 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.117800 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.117976 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.118352 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.118359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.119263 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:57.119863 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.119881 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.121746 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.121753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.123795 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.123816 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.124846 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.124851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.124990 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.125871 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:57.125892 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:57.127248 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:57.130735 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.130740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.130744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.131513 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.131950 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.131955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.131959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.132124 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.132981 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:57.132992 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:57.133250 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.133429 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.133797 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":478,"threads_started":1}
I20260810 19:31:57.140503 32177 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:33677
I20260810 19:31:57.140523 32325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:33677 every 8 connection(s)
I20260810 19:31:57.141674 32177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:31:57.142292 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 32177
I20260810 19:31:57.142554 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:31:57.143165 30630 external_mini_cluster.cc:1428] Setting key f04c1e7f299f774d33255b5dc7aadf7c
I20260810 19:31:57.144328 32326 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33175
I20260810 19:31:57.144588 32326 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:57.145361 32326 heartbeater.cc:511] Master 127.29.233.190:33175 requested a full tablet report, sending...
I20260810 19:31:57.146935 31931 ts_manager.cc:194] Registered new tserver with Master: 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677)
I20260810 19:31:57.147823 31931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:39987
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:31:57.193467 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33175
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:31:57.534978 32332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:31:57.535272 32332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:31:57.535431 32332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:31:57.542881 32332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:31:57.543007 32332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:31:57.543148 32332 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:31:57.543275 32332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:31:57.553527 32332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:31:57.554945 32332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:31:57.556385 32332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:31:57.570689 32338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:57.570713 32337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:31:57.570693 32341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:31:57.572129 32332 server_base.cc:1034] running on GCE node
I20260810 19:31:57.572983 32332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:31:57.574071 32332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:31:57.575344 32332 hybrid_clock.cc:648] HybridClock initialized: now 1786390317575248 us; error 68 us; skew 500 ppm
I20260810 19:31:57.575678 32332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:31:57 dist-test-slave-tm1g krb5kdc[31884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:31:57.586715 32332 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:31:57.588487 32332 webserver.cc:460] Webserver started at http://127.29.233.131:40035/ using document root <none> and password file <none>
I20260810 19:31:57.589309 32332 fs_manager.cc:359] Metadata directory not provided
I20260810 19:31:57.589538 32332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:31:57.589972 32332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:31:57.593178 32332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef"
format_stamp: "Formatted at 2026-08-10 19:31:57 on dist-test-slave-tm1g"
server_key: "55c5a1087b112b920d1de9612e14bd39"
server_key_iv: "68ab8982d575584fe92b1ba559f4c075"
server_key_version: "encryptionkey@0"
I20260810 19:31:57.594033 32332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef"
format_stamp: "Formatted at 2026-08-10 19:31:57 on dist-test-slave-tm1g"
server_key: "55c5a1087b112b920d1de9612e14bd39"
server_key_iv: "68ab8982d575584fe92b1ba559f4c075"
server_key_version: "encryptionkey@0"
I20260810 19:31:57.599730 32332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 19:31:57.603647 32348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:57.605166 32332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:31:57.605387 32332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef"
format_stamp: "Formatted at 2026-08-10 19:31:57 on dist-test-slave-tm1g"
server_key: "55c5a1087b112b920d1de9612e14bd39"
server_key_iv: "68ab8982d575584fe92b1ba559f4c075"
server_key_version: "encryptionkey@0"
I20260810 19:31:57.605620 32332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:31:57.612313 32173 heartbeater.cc:503] Master 127.29.233.190:33175 was elected leader, sending a full tablet report...
I20260810 19:31:57.622737 32332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:31:57.627394 32332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:31:57.627684 32332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:31:57.630642 32332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:31:57.630771 32332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:31:57.630908 32332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:31:57.630980 32332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:31:57 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:57.659430 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.632180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33175 (local address 127.29.233.131:41737)
0810 19:31:57.633139 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.633201 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.636253 (+ 3052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.637156 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.637172 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.638104 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:57.639221 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.639253 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.640470 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.640478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.643359 (+ 2881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.643378 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.644439 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.644448 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.644690 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.645666 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:57.645705 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:57.648694 (+ 2989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:57.653680 (+ 4986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.653687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.653699 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.654129 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.654629 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.654632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.654634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.654826 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.655574 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:57.655586 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:57.656205 (+ 619us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.656558 (+ 353us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.657285 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":347,"threads_started":1}
I20260810 19:31:57.659852 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.631659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:41737 (local address 127.29.233.190:33175)
0810 19:31:57.631901 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.631908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.633353 (+ 1445us) server_negotiation.cc:407] Connection header received
0810 19:31:57.636635 (+ 3282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.636641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.636726 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.636822 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:57.639430 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.640296 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.643557 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.644159 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.644356 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.648978 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:57.648988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:57.648996 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:57.649054 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:57.653522 (+ 4468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.654261 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.654270 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.654276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.654487 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.654953 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.654961 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.654967 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.655390 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:57.658490 (+ 3100us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.658717 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.659390 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:31:57.682986 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.667025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:35299 (local address 127.29.233.190:33175)
0810 19:31:57.667227 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.667235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.668114 (+ 879us) server_negotiation.cc:407] Connection header received
0810 19:31:57.668385 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.668390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.668469 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.668575 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:57.669491 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.670203 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.672293 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.673085 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.673253 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.675985 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:57.675994 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:57.676002 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:57.676078 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:57.679700 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.680750 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.680757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.680762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.680981 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.681482 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.681486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.681488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.681769 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:57.681901 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.682531 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.682717 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:31:57.683537 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.667115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33175 (local address 127.29.233.131:35299)
0810 19:31:57.667993 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.668017 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.668225 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.668597 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.668601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.669059 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:57.669346 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.669355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.670394 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.670399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.672128 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.672146 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.673278 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.673282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.673423 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.674146 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:57.674176 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:57.675688 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:57.679832 (+ 4144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.679844 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.679849 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.680608 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.681122 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:57.681131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:57.681138 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:57.681336 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:57.681905 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:57.681912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:57.682209 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.682413 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.682902 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":485,"threads_started":1}
I20260810 19:31:57.693087 32332 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:44933
I20260810 19:31:57.693111 32482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:44933 every 8 connection(s)
I20260810 19:31:57.694482 32332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:31:57.696849 32483 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33175
I20260810 19:31:57.696874 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 32332
I20260810 19:31:57.697079 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:31:57.697093 32483 heartbeater.cc:464] Registering TS with master...
I20260810 19:31:57.697521 30630 external_mini_cluster.cc:1428] Setting key 7fef8b22513b01b82737c34b043e9713
I20260810 19:31:57.697839 32483 heartbeater.cc:511] Master 127.29.233.190:33175 requested a full tablet report, sending...
I20260810 19:31:57.699517 31931 ts_manager.cc:194] Registered new tserver with Master: 3100c3a0b2ed41ee8bc00f6b23a87eef (127.29.233.131:44933)
I20260810 19:31:57.700773 31931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:41737
I20260810 19:31:57.701221 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 19:31:57 dist-test-slave-tm1g krb5kdc[31884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 19:31:57 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:57.761569 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.746389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.29.233.190:33175)
0810 19:31:57.747144 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.747152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.747177 (+ 25us) server_negotiation.cc:407] Connection header received
0810 19:31:57.747255 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.747259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.747339 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.747456 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:57.748959 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.749704 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.751099 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.751528 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.751670 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.755910 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:57.755917 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:57.755920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:57.755961 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:57.758664 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.759436 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.759444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.759450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.759603 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:57.760127 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:57.760131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:57.760133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:57.760469 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:57.760747 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.761077 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.761270 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597}
I20260810 19:31:57.766405 31931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:31:57.773296 31931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:31:57.800432 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.785804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46807 (local address 127.0.0.1:51104)
0810 19:31:57.789763 (+ 3959us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.789788 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.789932 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.793486 (+ 3554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.793491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.793516 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:57.793865 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.793875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.795729 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.795734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.797706 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.797718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.799228 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.799234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.799480 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.799553 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.799688 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.799924 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2518,"mutex_wait_us":49,"thread_start_us":1325,"threads_started":1}
I20260810 19:31:57.801067 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.785697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33677 (local address 127.0.0.1:53170)
0810 19:31:57.787525 (+ 1828us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.787577 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.787747 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.791556 (+ 3809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.791567 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.791611 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:57.792171 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.792189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.794431 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.794438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.796963 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.796979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.798976 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.798985 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.800391 (+ 1406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.800454 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.800550 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.800697 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1573,"mutex_wait_us":25,"thread_start_us":2412,"threads_started":1}
I20260810 19:31:57.801532 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.785813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53170 (local address 127.29.233.130:33677)
0810 19:31:57.790635 (+ 4822us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.790648 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.790686 (+ 38us) server_negotiation.cc:407] Connection header received
0810 19:31:57.790881 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.790894 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.791256 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.793028 (+ 1772us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:57.793247 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.794270 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.797278 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.798761 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.798949 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.799153 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.800972 (+ 1819us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.801162 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4602,"thread_start_us":4405,"threads_started":1}
I20260810 19:31:57.801512 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.786366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.29.233.129:46807)
0810 19:31:57.792896 (+ 6530us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.792903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.792928 (+ 25us) server_negotiation.cc:407] Connection header received
0810 19:31:57.793026 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.793034 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.793248 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.794055 (+ 807us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:57.794243 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.795595 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.797858 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.798915 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.800760 (+ 1845us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.800938 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.801007 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.801191 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6359,"thread_start_us":6214,"threads_started":1}
I20260810 19:31:57.802270 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.789307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44933 (local address 127.0.0.1:50308)
0810 19:31:57.790910 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.790933 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.791081 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.792178 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.792182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.792208 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:57.792644 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.792658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.794815 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.794820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.797075 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.797089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.801316 (+ 4227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.801324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.801571 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.801601 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.801691 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.801822 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1462,"mutex_wait_us":1,"thread_start_us":1327,"threads_started":1}
I20260810 19:31:57.802793 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.788959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50308 (local address 127.29.233.131:44933)
0810 19:31:57.791738 (+ 2779us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.791743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.791769 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:31:57.791842 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.791847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.792000 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.792195 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:57.792899 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.794673 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.797209 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.798761 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.798985 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.799153 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.802231 (+ 3078us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.802467 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2625,"thread_start_us":2434,"threads_started":1}
I20260810 19:31:57.805243 32250 tablet_service.cc:1467] Processing CreateTablet for tablet d971e890f2d84346b3dd9b77d3ade8a9 (DEFAULT_TABLE table=test-table [id=d04f2603e764414bb354b30450ddfce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:57.805956 32093 tablet_service.cc:1467] Processing CreateTablet for tablet d971e890f2d84346b3dd9b77d3ade8a9 (DEFAULT_TABLE table=test-table [id=d04f2603e764414bb354b30450ddfce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:57.806087 32403 tablet_service.cc:1467] Processing CreateTablet for tablet d971e890f2d84346b3dd9b77d3ade8a9 (DEFAULT_TABLE table=test-table [id=d04f2603e764414bb354b30450ddfce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:57.806636 32250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d971e890f2d84346b3dd9b77d3ade8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:57.807376 32093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d971e890f2d84346b3dd9b77d3ade8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:57.807394 32403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d971e890f2d84346b3dd9b77d3ade8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:57.819365 32504 tablet_bootstrap.cc:492] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Bootstrap starting.
I20260810 19:31:57.822278 32505 tablet_bootstrap.cc:492] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Bootstrap starting.
I20260810 19:31:57.822966 32504 tablet_bootstrap.cc:654] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:57.824841 32504 log.cc:824] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:57.826185 32505 tablet_bootstrap.cc:654] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:57.827577 32505 log.cc:824] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:57.828330 32506 tablet_bootstrap.cc:492] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Bootstrap starting.
I20260810 19:31:57.828387 32504 tablet_bootstrap.cc:492] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: No bootstrap required, opened a new log
I20260810 19:31:57.828670 32504 ts_tablet_manager.cc:1397] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 19:31:57.832168 32505 tablet_bootstrap.cc:492] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: No bootstrap required, opened a new log
I20260810 19:31:57.832507 32505 ts_tablet_manager.cc:1397] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260810 19:31:57.833278 32506 tablet_bootstrap.cc:654] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:57.834650 32506 log.cc:824] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Log is configured to *not* fsync() on all Append() calls
I20260810 19:31:57.838943 32506 tablet_bootstrap.cc:492] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: No bootstrap required, opened a new log
I20260810 19:31:57.839238 32506 ts_tablet_manager.cc:1397] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.008s
I20260810 19:31:57.840639 32505 raft_consensus.cc:348] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.840552 32504 raft_consensus.cc:348] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.841005 32504 raft_consensus.cc:374] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:57.841013 32505 raft_consensus.cc:374] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:57.841114 32505 raft_consensus.cc:729] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3100c3a0b2ed41ee8bc00f6b23a87eef, State: Initialized, Role: FOLLOWER
I20260810 19:31:57.841140 32504 raft_consensus.cc:729] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d93c6c47894b1ca1f8e26adbabc958, State: Initialized, Role: FOLLOWER
I20260810 19:31:57.841888 32504 consensus_queue.cc:260] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.842197 32505 consensus_queue.cc:260] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.842630 32510 raft_consensus.cc:482] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:31:57.843030 32510 raft_consensus.cc:504] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.846748 32510 leader_election.cc:288] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677), cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807)
I20260810 19:31:57.847564 32483 heartbeater.cc:503] Master 127.29.233.190:33175 was elected leader, sending a full tablet report...
I20260810 19:31:57.851953 32506 raft_consensus.cc:348] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.852463 32506 raft_consensus.cc:374] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:57.852576 32506 raft_consensus.cc:729] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff8b485a6ca4ddc97916c6b65aaef94, State: Initialized, Role: FOLLOWER
I20260810 19:31:57.853231 32506 consensus_queue.cc:260] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.855465 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.846047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33677 (local address 127.29.233.131:49349)
0810 19:31:57.846361 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.846392 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.846540 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.848623 (+ 2083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.848629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.848659 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:57.848896 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.848908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.850382 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.850388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.852393 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.852408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.854316 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.854325 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.854655 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.854698 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.854818 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.855029 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 19:31:57.855271 32505 ts_tablet_manager.cc:1428] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Time spent starting tablet: real 0.023s user 0.006s sys 0.018s
I20260810 19:31:57.856904 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.846688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:49349 (local address 127.29.233.130:33677)
0810 19:31:57.847573 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.847582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.847603 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:31:57.847675 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.847681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.847753 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.847850 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:57.849081 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.850214 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.852637 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.853885 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.854063 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.854186 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.855364 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.855582 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726}
I20260810 19:31:57.857272 32506 ts_tablet_manager.cc:1428] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20260810 19:31:57.858006 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d971e890f2d84346b3dd9b77d3ade8a9" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d93c6c47894b1ca1f8e26adbabc958" is_pre_election: true
I20260810 19:31:57.858631 32270 raft_consensus.cc:2454] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 0.
I20260810 19:31:57.859498 32350 leader_election.cc:302] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3100c3a0b2ed41ee8bc00f6b23a87eef, 86d93c6c47894b1ca1f8e26adbabc958; no voters:
I20260810 19:31:57.861011 32510 raft_consensus.cc:2781] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:31:57.861200 32510 raft_consensus.cc:482] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:31:57.861315 32510 raft_consensus.cc:3037] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:57.864835 32510 raft_consensus.cc:504] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.865602 32510 leader_election.cc:288] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 election: Requested vote from peers 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677), cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807)
I20260810 19:31:57.866019 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d971e890f2d84346b3dd9b77d3ade8a9" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d93c6c47894b1ca1f8e26adbabc958"
I20260810 19:31:57.866240 32270 raft_consensus.cc:3037] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:57.866340 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.848324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:34079 (local address 127.29.233.129:46807)
0810 19:31:57.848537 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.848543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.855918 (+ 7375us) server_negotiation.cc:407] Connection header received
0810 19:31:57.856137 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.856143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.856228 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.856322 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:31:57.858131 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.859602 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.862843 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.864651 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.864825 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.864936 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.865765 (+ 829us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.865971 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:31:57.867151 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.847169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46807 (local address 127.29.233.131:34079)
0810 19:31:57.855823 (+ 8654us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:57.855842 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:57.855986 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:57.857601 (+ 1615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:57.857609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:57.857648 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:31:57.857918 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.857932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.859838 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.859842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.862583 (+ 2741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:57.862609 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.865158 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.865167 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:57.865484 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.865553 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 19:31:57.866568 (+ 1015us) client_negotiation.cc:240] Negotiation successful
0810 19:31:57.866737 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8552,"thread_start_us":19676,"threads_started":1}
I20260810 19:31:57.867746 32326 heartbeater.cc:503] Master 127.29.233.190:33175 was elected leader, sending a full tablet report...
I20260810 19:31:57.868294 32504 ts_tablet_manager.cc:1428] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Time spent starting tablet: real 0.040s user 0.014s sys 0.004s
I20260810 19:31:57.869735 32270 raft_consensus.cc:2454] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 1.
I20260810 19:31:57.869485 32113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d971e890f2d84346b3dd9b77d3ade8a9" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff8b485a6ca4ddc97916c6b65aaef94"
I20260810 19:31:57.869550 32114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d971e890f2d84346b3dd9b77d3ade8a9" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" is_pre_election: true
I20260810 19:31:57.870157 32114 raft_consensus.cc:2454] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 0.
I20260810 19:31:57.872051 32350 leader_election.cc:302] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3100c3a0b2ed41ee8bc00f6b23a87eef, 86d93c6c47894b1ca1f8e26adbabc958; no voters: cff8b485a6ca4ddc97916c6b65aaef94
I20260810 19:31:57.872537 32510 raft_consensus.cc:2781] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:31:57.872720 32510 raft_consensus.cc:686] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 LEADER]: Becoming Leader. State: Replica: 3100c3a0b2ed41ee8bc00f6b23a87eef, State: Running, Role: LEADER
I20260810 19:31:57.873206 32510 consensus_queue.cc:237] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:57.877632 31931 catalog_manager.cc:5495] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3100c3a0b2ed41ee8bc00f6b23a87eef (127.29.233.131). New cstate: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } health_report { overall_health: HEALTHY } } }
W20260810 19:31:57.894001 32328 tablet.cc:2369] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:31:57.911716 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:57.905439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.29.233.131:44933)
0810 19:31:57.905642 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:57.905647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:57.905842 (+ 195us) server_negotiation.cc:407] Connection header received
0810 19:31:57.906104 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:57.906109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:57.906191 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:57.906416 (+ 225us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:31:57.906748 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.907502 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.909015 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:57.909598 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:57.909726 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:57.910423 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:31:57.911099 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:31:57.911225 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:57.911361 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 19:31:57.911480 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:31:57.920113 32114 raft_consensus.cc:3037] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:57.920362 32270 raft_consensus.cc:1261] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Refusing update from remote peer 3100c3a0b2ed41ee8bc00f6b23a87eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:31:57.921307 32512 consensus_queue.cc:1035] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:31:57.923357 32114 raft_consensus.cc:1261] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Refusing update from remote peer 3100c3a0b2ed41ee8bc00f6b23a87eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:31:57.924207 32512 consensus_queue.cc:1035] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:31:57.951857 31931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:31:57.952145 31931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:31:57.956708 31931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94: Sending DeleteTablet for 3 replicas of tablet d971e890f2d84346b3dd9b77d3ade8a9
I20260810 19:31:57.958105 32403 tablet_service.cc:1514] Processing DeleteTablet for tablet d971e890f2d84346b3dd9b77d3ade8a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:31:57 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:50308
I20260810 19:31:57.958173 32250 tablet_service.cc:1514] Processing DeleteTablet for tablet d971e890f2d84346b3dd9b77d3ade8a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:31:57 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:53170
I20260810 19:31:57.958276 32093 tablet_service.cc:1514] Processing DeleteTablet for tablet d971e890f2d84346b3dd9b77d3ade8a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:31:57 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51104
I20260810 19:31:57.960425 31933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:31:57.961141 31933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:31:57.970976 32249 tablet_service.cc:1467] Processing CreateTablet for tablet e7a3a93c35a243f39f1d03679f96c0cb (DEFAULT_TABLE table=test-table [id=11d4d4fc59ce4a74909051a8de806480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:57.970989 32402 tablet_service.cc:1467] Processing CreateTablet for tablet e7a3a93c35a243f39f1d03679f96c0cb (DEFAULT_TABLE table=test-table [id=11d4d4fc59ce4a74909051a8de806480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:57.971696 32402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7a3a93c35a243f39f1d03679f96c0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:57.971710 32249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7a3a93c35a243f39f1d03679f96c0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:57.975384 32091 tablet_service.cc:1467] Processing CreateTablet for tablet e7a3a93c35a243f39f1d03679f96c0cb (DEFAULT_TABLE table=test-table [id=11d4d4fc59ce4a74909051a8de806480]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:31:57.976173 32091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7a3a93c35a243f39f1d03679f96c0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:57.987372 32504 tablet_bootstrap.cc:492] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: Bootstrap starting.
I20260810 19:31:57.989136 32527 tablet_replica.cc:330] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: stopping tablet replica
I20260810 19:31:57.989646 32527 raft_consensus.cc:2229] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:31:57.989787 32504 tablet_bootstrap.cc:654] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:57.990232 32527 raft_consensus.cc:2258] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:31:57.991005 32505 tablet_bootstrap.cc:492] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Bootstrap starting.
I20260810 19:31:57.992821 32505 tablet_bootstrap.cc:654] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:57.993598 32530 tablet_replica.cc:330] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: stopping tablet replica
I20260810 19:31:57.994551 32506 tablet_bootstrap.cc:492] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Bootstrap starting.
I20260810 19:31:57.999529 32530 raft_consensus.cc:2229] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:31:58.000993 32506 tablet_bootstrap.cc:654] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:58.001693 32530 raft_consensus.cc:2258] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:31:58.004618 32505 tablet_bootstrap.cc:492] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: No bootstrap required, opened a new log
I20260810 19:31:58.004833 32505 ts_tablet_manager.cc:1397] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260810 19:31:58.005795 32505 raft_consensus.cc:348] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.006050 32505 raft_consensus.cc:374] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:58.006184 32505 raft_consensus.cc:729] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3100c3a0b2ed41ee8bc00f6b23a87eef, State: Initialized, Role: FOLLOWER
I20260810 19:31:58.006567 32505 consensus_queue.cc:260] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.011670 32505 ts_tablet_manager.cc:1428] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260810 19:31:58.013319 32504 tablet_bootstrap.cc:492] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: No bootstrap required, opened a new log
I20260810 19:31:58.013490 32504 ts_tablet_manager.cc:1397] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.013s
I20260810 19:31:58.014526 32504 raft_consensus.cc:348] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.014770 32504 raft_consensus.cc:374] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:58.014922 32504 raft_consensus.cc:729] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d93c6c47894b1ca1f8e26adbabc958, State: Initialized, Role: FOLLOWER
I20260810 19:31:58.015098 32526 tablet_replica.cc:330] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: stopping tablet replica
I20260810 19:31:58.015259 32504 consensus_queue.cc:260] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.015591 32526 raft_consensus.cc:2229] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:31:58.016218 32526 raft_consensus.cc:2258] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:31:58.016422 32530 ts_tablet_manager.cc:1905] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:31:58.016752 32504 ts_tablet_manager.cc:1428] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 19:31:58.020048 32527 ts_tablet_manager.cc:1905] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:31:58.020387 32506 tablet_bootstrap.cc:492] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: No bootstrap required, opened a new log
I20260810 19:31:58.020547 32506 ts_tablet_manager.cc:1397] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.003s
I20260810 19:31:58.021381 32506 raft_consensus.cc:348] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.021624 32506 raft_consensus.cc:374] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:58.021725 32506 raft_consensus.cc:729] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff8b485a6ca4ddc97916c6b65aaef94, State: Initialized, Role: FOLLOWER
I20260810 19:31:58.022054 32506 consensus_queue.cc:260] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.022796 32506 ts_tablet_manager.cc:1428] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 19:31:58.027400 32527 ts_tablet_manager.cc:1918] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:31:58.027712 32527 log.cc:1196] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/d971e890f2d84346b3dd9b77d3ade8a9
I20260810 19:31:58.027833 32530 ts_tablet_manager.cc:1918] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:31:58.028014 32530 log.cc:1196] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/d971e890f2d84346b3dd9b77d3ade8a9
I20260810 19:31:58.028339 32527 ts_tablet_manager.cc:1939] T d971e890f2d84346b3dd9b77d3ade8a9 P 86d93c6c47894b1ca1f8e26adbabc958: Deleting consensus metadata
I20260810 19:31:58.028703 32530 ts_tablet_manager.cc:1939] T d971e890f2d84346b3dd9b77d3ade8a9 P cff8b485a6ca4ddc97916c6b65aaef94: Deleting consensus metadata
I20260810 19:31:58.029032 32526 ts_tablet_manager.cc:1905] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:31:58.030369 31916 catalog_manager.cc:4841] TS 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677): tablet d971e890f2d84346b3dd9b77d3ade8a9 (table test-table [id=d04f2603e764414bb354b30450ddfce1]) successfully deleted
I20260810 19:31:58.030478 31914 catalog_manager.cc:4841] TS cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807): tablet d971e890f2d84346b3dd9b77d3ade8a9 (table test-table [id=d04f2603e764414bb354b30450ddfce1]) successfully deleted
I20260810 19:31:58.036947 32526 ts_tablet_manager.cc:1918] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:31:58.037154 32526 log.cc:1196] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/d971e890f2d84346b3dd9b77d3ade8a9
I20260810 19:31:58.037716 32526 ts_tablet_manager.cc:1939] T d971e890f2d84346b3dd9b77d3ade8a9 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Deleting consensus metadata
I20260810 19:31:58.039609 31916 catalog_manager.cc:4841] TS 3100c3a0b2ed41ee8bc00f6b23a87eef (127.29.233.131:44933): tablet d971e890f2d84346b3dd9b77d3ade8a9 (table test-table [id=d04f2603e764414bb354b30450ddfce1]) successfully deleted
W20260810 19:31:58.105127 32174 tablet.cc:2369] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:31:58.197636 32485 tablet.cc:2369] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:31:58.288331 32510 raft_consensus.cc:482] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:31:58.288652 32510 raft_consensus.cc:504] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.289526 32510 leader_election.cc:288] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807), 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677)
I20260810 19:31:58.290251 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a3a93c35a243f39f1d03679f96c0cb" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d93c6c47894b1ca1f8e26adbabc958" is_pre_election: true
I20260810 19:31:58.290249 32114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a3a93c35a243f39f1d03679f96c0cb" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" is_pre_election: true
I20260810 19:31:58.290558 32114 raft_consensus.cc:2454] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 0.
I20260810 19:31:58.290557 32270 raft_consensus.cc:2454] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 0.
I20260810 19:31:58.291067 32349 leader_election.cc:302] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3100c3a0b2ed41ee8bc00f6b23a87eef, cff8b485a6ca4ddc97916c6b65aaef94; no voters:
I20260810 19:31:58.291435 32510 raft_consensus.cc:2781] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:31:58.291556 32510 raft_consensus.cc:482] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:31:58.291710 32510 raft_consensus.cc:3037] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:58.295186 32510 raft_consensus.cc:504] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.295941 32510 leader_election.cc:288] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 election: Requested vote from peers cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807), 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677)
I20260810 19:31:58.296564 32114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a3a93c35a243f39f1d03679f96c0cb" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff8b485a6ca4ddc97916c6b65aaef94"
I20260810 19:31:58.296609 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a3a93c35a243f39f1d03679f96c0cb" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d93c6c47894b1ca1f8e26adbabc958"
I20260810 19:31:58.296813 32114 raft_consensus.cc:3037] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:58.296844 32270 raft_consensus.cc:3037] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:58.299345 32114 raft_consensus.cc:2454] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 1.
I20260810 19:31:58.299841 32349 leader_election.cc:302] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3100c3a0b2ed41ee8bc00f6b23a87eef, cff8b485a6ca4ddc97916c6b65aaef94; no voters:
I20260810 19:31:58.299998 32270 raft_consensus.cc:2454] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 1.
I20260810 19:31:58.300189 32510 raft_consensus.cc:2781] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:31:58.300453 32510 raft_consensus.cc:686] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 LEADER]: Becoming Leader. State: Replica: 3100c3a0b2ed41ee8bc00f6b23a87eef, State: Running, Role: LEADER
I20260810 19:31:58.300913 32510 consensus_queue.cc:237] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.304083 31931 catalog_manager.cc:5495] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3100c3a0b2ed41ee8bc00f6b23a87eef (127.29.233.131). New cstate: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } health_report { overall_health: HEALTHY } } }
Aug 10 19:31:58 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390315, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:31:58.398905 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.381297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.29.233.190:33175)
0810 19:31:58.382454 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.382473 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.382834 (+ 361us) server_negotiation.cc:407] Connection header received
0810 19:31:58.382998 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.383005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.383091 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.383223 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:58.384380 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.385588 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.386997 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.387676 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.387809 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.391798 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:58.391806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:58.391811 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:58.391850 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:58.395661 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.396486 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.396492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.396496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.396654 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.397139 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.397144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.397147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.397514 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:58.397657 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.398136 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.398384 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":687,"threads_started":1}
I20260810 19:31:58.399019 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.381297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33175 (local address 127.0.0.1:52066)
0810 19:31:58.382718 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:58.382735 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:58.382878 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:58.383278 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:58.383282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:58.383991 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:31:58.384205 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:58.384214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.385750 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.385754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:58.386843 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:58.386856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.387890 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.387895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:58.388055 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.388854 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:31:58.388874 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:31:58.391569 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:31:58.395837 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:58.395851 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:58.395857 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:58.396328 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:58.396824 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:31:58.396828 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:31:58.396831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:31:58.397019 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:31:58.397667 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:31:58.397674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:31:58.397845 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 19:31:58.397984 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 19:31:58.398222 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1277,"thread_start_us":1114,"threads_started":1}
I20260810 19:31:58.404984 31931 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52066:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 19:31:58.406411 31931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:31:58.414219 32093 tablet_service.cc:1467] Processing CreateTablet for tablet 25b6a9dd78284b05b53c93cf0229e1f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74842219af394ca58c52d9903c792053]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:31:58.414587 32249 tablet_service.cc:1467] Processing CreateTablet for tablet 25b6a9dd78284b05b53c93cf0229e1f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74842219af394ca58c52d9903c792053]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:31:58.415009 32093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b6a9dd78284b05b53c93cf0229e1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:58.415186 32249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b6a9dd78284b05b53c93cf0229e1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:58.419293 32403 tablet_service.cc:1467] Processing CreateTablet for tablet 25b6a9dd78284b05b53c93cf0229e1f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74842219af394ca58c52d9903c792053]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:31:58.419907 32403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b6a9dd78284b05b53c93cf0229e1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:31:58.422255 32506 tablet_bootstrap.cc:492] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: Bootstrap starting.
I20260810 19:31:58.424132 32506 tablet_bootstrap.cc:654] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:58.425876 32504 tablet_bootstrap.cc:492] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: Bootstrap starting.
I20260810 19:31:58.427585 32504 tablet_bootstrap.cc:654] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:58.428632 32505 tablet_bootstrap.cc:492] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Bootstrap starting.
I20260810 19:31:58.429659 32506 tablet_bootstrap.cc:492] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: No bootstrap required, opened a new log
I20260810 19:31:58.429870 32506 ts_tablet_manager.cc:1397] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 19:31:58.430266 32505 tablet_bootstrap.cc:654] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Neither blocks nor log segments found. Creating new log.
I20260810 19:31:58.430552 32504 tablet_bootstrap.cc:492] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: No bootstrap required, opened a new log
I20260810 19:31:58.430727 32504 ts_tablet_manager.cc:1397] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 19:31:58.430724 32506 raft_consensus.cc:348] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.430999 32506 raft_consensus.cc:374] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:58.431083 32506 raft_consensus.cc:729] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff8b485a6ca4ddc97916c6b65aaef94, State: Initialized, Role: FOLLOWER
I20260810 19:31:58.431408 32506 consensus_queue.cc:260] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.431550 32504 raft_consensus.cc:348] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.431885 32504 raft_consensus.cc:374] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:58.432011 32504 raft_consensus.cc:729] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d93c6c47894b1ca1f8e26adbabc958, State: Initialized, Role: FOLLOWER
I20260810 19:31:58.431998 32514 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.432328 32506 ts_tablet_manager.cc:1428] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 19:31:58.432418 32514 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:31:58.432301 32504 consensus_queue.cc:260] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.432842 32504 ts_tablet_manager.cc:1428] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 19:31:58.432786 32511 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.433121 32511 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:31:58.433624 32505 tablet_bootstrap.cc:492] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: No bootstrap required, opened a new log
I20260810 19:31:58.433817 32505 ts_tablet_manager.cc:1397] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 19:31:58.434701 32505 raft_consensus.cc:348] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.434986 32505 raft_consensus.cc:374] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:31:58.435068 32505 raft_consensus.cc:729] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3100c3a0b2ed41ee8bc00f6b23a87eef, State: Initialized, Role: FOLLOWER
I20260810 19:31:58.435330 32505 consensus_queue.cc:260] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.435848 32510 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.435988 32505 ts_tablet_manager.cc:1428] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 19:31:58.436102 32510 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:31:58.499577 32510 raft_consensus.cc:482] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:31:58.499766 32510 raft_consensus.cc:504] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.500581 32510 leader_election.cc:288] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807), 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677)
I20260810 19:31:58.501060 32114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b6a9dd78284b05b53c93cf0229e1f4" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" is_pre_election: true
I20260810 19:31:58.501134 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b6a9dd78284b05b53c93cf0229e1f4" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d93c6c47894b1ca1f8e26adbabc958" is_pre_election: true
I20260810 19:31:58.501379 32114 raft_consensus.cc:2454] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 0.
I20260810 19:31:58.501400 32270 raft_consensus.cc:2454] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 0.
I20260810 19:31:58.501860 32349 leader_election.cc:302] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3100c3a0b2ed41ee8bc00f6b23a87eef, cff8b485a6ca4ddc97916c6b65aaef94; no voters:
I20260810 19:31:58.502195 32510 raft_consensus.cc:2781] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:31:58.502365 32510 raft_consensus.cc:482] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:31:58.502447 32510 raft_consensus.cc:3037] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:58.504554 32510 raft_consensus.cc:504] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.505142 32510 leader_election.cc:288] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 election: Requested vote from peers cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807), 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677)
I20260810 19:31:58.505637 32114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b6a9dd78284b05b53c93cf0229e1f4" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff8b485a6ca4ddc97916c6b65aaef94"
I20260810 19:31:58.505707 32270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b6a9dd78284b05b53c93cf0229e1f4" candidate_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d93c6c47894b1ca1f8e26adbabc958"
I20260810 19:31:58.505882 32114 raft_consensus.cc:3037] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:58.505996 32270 raft_consensus.cc:3037] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:31:58.508001 32270 raft_consensus.cc:2454] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 1.
I20260810 19:31:58.508002 32114 raft_consensus.cc:2454] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3100c3a0b2ed41ee8bc00f6b23a87eef in term 1.
I20260810 19:31:58.508440 32349 leader_election.cc:302] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3100c3a0b2ed41ee8bc00f6b23a87eef, cff8b485a6ca4ddc97916c6b65aaef94; no voters:
I20260810 19:31:58.508845 32510 raft_consensus.cc:2781] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:31:58.509019 32510 raft_consensus.cc:686] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 LEADER]: Becoming Leader. State: Replica: 3100c3a0b2ed41ee8bc00f6b23a87eef, State: Running, Role: LEADER
I20260810 19:31:58.509325 32510 consensus_queue.cc:237] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } }
I20260810 19:31:58.511049 32512 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: TxnStatusTablet state changed. Reason: New leader 3100c3a0b2ed41ee8bc00f6b23a87eef. Latest consensus state: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.511289 32512 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:31:58.512916 31933 catalog_manager.cc:5495] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3100c3a0b2ed41ee8bc00f6b23a87eef (127.29.233.131). New cstate: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } health_report { overall_health: HEALTHY } } }
I20260810 19:31:58.518805 32546 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 19:31:58.519137 32546 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 19:31:58.535827 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.529675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44933 (local address 127.0.0.1:49834)
0810 19:31:58.530001 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:58.530027 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:58.530204 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:58.530765 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:58.530769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:58.530779 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:31:58.531014 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:58.531031 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.532644 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.532648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:58.533649 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:58.533660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.534721 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.534726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:58.534878 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.534936 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:31:58.535382 (+ 446us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:31:58.535398 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 19:31:58.535460 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 19:31:58.535586 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 19:31:58.535997 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.529671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.29.233.131:44933)
0810 19:31:58.530404 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.530411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.530440 (+ 29us) server_negotiation.cc:407] Connection header received
0810 19:31:58.530522 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.530527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.530616 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.530736 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:31:58.531204 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.532485 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.533792 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.534562 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.534694 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.535132 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:31:58.535280 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:31:58.535375 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.535523 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.535606 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":356,"threads_started":1}
I20260810 19:31:58.540673 32114 raft_consensus.cc:1261] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Refusing update from remote peer 3100c3a0b2ed41ee8bc00f6b23a87eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:31:58.540846 32270 raft_consensus.cc:1261] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Refusing update from remote peer 3100c3a0b2ed41ee8bc00f6b23a87eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:31:58.541405 32512 consensus_queue.cc:1035] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:31:58.541704 32510 consensus_queue.cc:1035] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:31:58.547833 32514 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: TxnStatusTablet state changed. Reason: New leader 3100c3a0b2ed41ee8bc00f6b23a87eef. Latest consensus state: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.548106 32514 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:31:58.551685 32551 mvcc.cc:205] Tried to move back new op lower bound from 7317054744735903744 to 7317054744616886272. Current Snapshot: MvccSnapshot[applied={T|T < 7317054744735903744}]
I20260810 19:31:58.555828 32510 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.556178 32510 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:31:58.559912 32514 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.560207 32514 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P cff8b485a6ca4ddc97916c6b65aaef94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:31:58.563944 32511 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: TxnStatusTablet state changed. Reason: New leader 3100c3a0b2ed41ee8bc00f6b23a87eef. Latest consensus state: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.564190 32511 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:31:58.565886 32512 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.566162 32512 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P 3100c3a0b2ed41ee8bc00f6b23a87eef: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:31:58.569948 32511 tablet_replica.cc:439] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 } } peers { permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 } } peers { permanent_uuid: "3100c3a0b2ed41ee8bc00f6b23a87eef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44933 } } }
I20260810 19:31:58.570223 32511 tablet_replica.cc:442] T 25b6a9dd78284b05b53c93cf0229e1f4 P 86d93c6c47894b1ca1f8e26adbabc958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:31:58.584332 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.577531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44933 (local address 127.29.233.131:43709)
0810 19:31:58.578152 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 19:31:58.578170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:31:58.578294 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:31:58.578725 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:31:58.578728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:31:58.578737 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:31:58.579081 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:58.579089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.580745 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.580751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:58.582100 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:31:58.582112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.583143 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.583147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:31:58.583278 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.583332 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:31:58.583782 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:31:58.583789 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 19:31:58.583849 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 19:31:58.583960 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":20,"thread_start_us":356,"threads_started":1}
I20260810 19:31:58.584331 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.577550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:43709 (local address 127.29.233.131:44933)
0810 19:31:58.577721 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.577726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.578262 (+ 536us) server_negotiation.cc:407] Connection header received
0810 19:31:58.578518 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.578525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.578605 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.578718 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:31:58.579223 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.580469 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.582236 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.582982 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.583113 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.583531 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:31:58.583675 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:31:58.583778 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.583904 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.583984 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":53}
I20260810 19:31:58.595510 32270 raft_consensus.cc:1261] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Refusing update from remote peer 3100c3a0b2ed41ee8bc00f6b23a87eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:31:58.595602 32114 raft_consensus.cc:1261] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Refusing update from remote peer 3100c3a0b2ed41ee8bc00f6b23a87eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:31:58.596089 32563 consensus_queue.cc:1035] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d93c6c47894b1ca1f8e26adbabc958" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:31:58.596457 32512 consensus_queue.cc:1035] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff8b485a6ca4ddc97916c6b65aaef94" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:31:58.680728 31933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:31:58.681051 31933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:31:58.684300 31933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae0dcfde099a44a797f33abc6bd44a94: Sending DeleteTablet for 3 replicas of tablet e7a3a93c35a243f39f1d03679f96c0cb
I20260810 19:31:58.685159 32093 tablet_service.cc:1514] Processing DeleteTablet for tablet e7a3a93c35a243f39f1d03679f96c0cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:31:58 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51104
I20260810 19:31:58.685387 32249 tablet_service.cc:1514] Processing DeleteTablet for tablet e7a3a93c35a243f39f1d03679f96c0cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:31:58 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:53170
I20260810 19:31:58.686206 32403 tablet_service.cc:1514] Processing DeleteTablet for tablet e7a3a93c35a243f39f1d03679f96c0cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:31:58 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:50308
I20260810 19:31:58.694621 32570 tablet_replica.cc:330] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: stopping tablet replica
I20260810 19:31:58.694883 32570 raft_consensus.cc:2229] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:31:58.695236 32570 raft_consensus.cc:2258] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:31:58.697842 32570 ts_tablet_manager.cc:1905] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:31:58.699342 32572 tablet_replica.cc:330] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: stopping tablet replica
I20260810 19:31:58.699708 32572 raft_consensus.cc:2229] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:31:58.700343 32572 raft_consensus.cc:2258] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:31:58.701711 32572 ts_tablet_manager.cc:1905] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:31:58.709623 32572 ts_tablet_manager.cc:1918] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:31:58.709875 32572 log.cc:1196] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/e7a3a93c35a243f39f1d03679f96c0cb
I20260810 19:31:58.710703 32572 ts_tablet_manager.cc:1939] T e7a3a93c35a243f39f1d03679f96c0cb P 3100c3a0b2ed41ee8bc00f6b23a87eef: Deleting consensus metadata
I20260810 19:31:58.712103 31916 catalog_manager.cc:4841] TS 3100c3a0b2ed41ee8bc00f6b23a87eef (127.29.233.131:44933): tablet e7a3a93c35a243f39f1d03679f96c0cb (table test-table [id=11d4d4fc59ce4a74909051a8de806480]) successfully deleted
I20260810 19:31:58.712788 32570 ts_tablet_manager.cc:1918] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:31:58.712934 32570 log.cc:1196] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/e7a3a93c35a243f39f1d03679f96c0cb
I20260810 19:31:58.713425 32570 ts_tablet_manager.cc:1939] T e7a3a93c35a243f39f1d03679f96c0cb P 86d93c6c47894b1ca1f8e26adbabc958: Deleting consensus metadata
I20260810 19:31:58.714599 31916 catalog_manager.cc:4841] TS 86d93c6c47894b1ca1f8e26adbabc958 (127.29.233.130:33677): tablet e7a3a93c35a243f39f1d03679f96c0cb (table test-table [id=11d4d4fc59ce4a74909051a8de806480]) successfully deleted
I20260810 19:31:58.718392 32569 tablet_replica.cc:330] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: stopping tablet replica
I20260810 19:31:58.718693 32569 raft_consensus.cc:2229] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:31:58.719046 32569 raft_consensus.cc:2258] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:31:58.721817 32569 ts_tablet_manager.cc:1905] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:31:58.732024 32569 ts_tablet_manager.cc:1918] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:31:58.732282 32569 log.cc:1196] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/e7a3a93c35a243f39f1d03679f96c0cb
I20260810 19:31:58.732879 32569 ts_tablet_manager.cc:1939] T e7a3a93c35a243f39f1d03679f96c0cb P cff8b485a6ca4ddc97916c6b65aaef94: Deleting consensus metadata
I20260810 19:31:58.734650 31914 catalog_manager.cc:4841] TS cff8b485a6ca4ddc97916c6b65aaef94 (127.29.233.129:46807): tablet e7a3a93c35a243f39f1d03679f96c0cb (table test-table [id=11d4d4fc59ce4a74909051a8de806480]) successfully deleted
Aug 10 19:31:58 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:31:58.748867 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.715408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.29.233.129:46807)
0810 19:31:58.729699 (+ 14291us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.729703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.729725 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:31:58.729777 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.729781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.729851 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.730011 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:58.732042 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.733371 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.735287 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.735976 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.736113 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.741323 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:58.741340 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:58.741361 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:58.741411 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:58.745620 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.746492 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.746503 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.746507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.746682 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.747077 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.747083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.747087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.747538 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:58.747849 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.748163 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.748403 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14153,"thread_start_us":13973,"threads_started":1}
W20260810 19:31:58.750114 32167 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47170
I20260810 19:31:58.771826 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.753442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52068 (local address 127.29.233.190:33175)
0810 19:31:58.753637 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.753643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.758934 (+ 5291us) server_negotiation.cc:407] Connection header received
0810 19:31:58.759167 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.759172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.759257 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.759375 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:58.760801 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.761579 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.763049 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.763576 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.763708 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.766116 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:58.766129 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:58.766132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:58.766166 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:58.769059 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.769749 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.769755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.769758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.769893 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.770287 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.770292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.770297 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.770642 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:58.770902 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.771262 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.771509 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 19:31:58.772432 31933 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52068
I20260810 19:31:58.774811 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32016
I20260810 19:31:58.790706 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.775797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47172 (local address 127.29.233.129:46807)
0810 19:31:58.776032 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.776036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.778766 (+ 2730us) server_negotiation.cc:407] Connection header received
0810 19:31:58.778971 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.778976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.779072 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.779181 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:58.780322 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.781155 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.782536 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.782996 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.783141 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.785488 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:58.785493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:58.785497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:58.785529 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:58.788074 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.788730 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.788736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.788740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.788867 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.789251 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.789257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.789260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.789535 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:58.789800 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.790111 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.790373 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 19:31:58.791301 32167 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47172
W20260810 19:31:58.791947 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32016: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:31:58.792176 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 32016
I20260810 19:31:58.821477 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32177
Aug 10 19:31:58 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:31:58.837960 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.822420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58306 (local address 127.29.233.130:33677)
0810 19:31:58.822982 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.822987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.823003 (+ 16us) server_negotiation.cc:407] Connection header received
0810 19:31:58.823058 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.823061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.823123 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.823246 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:58.824472 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.825330 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.826705 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.827141 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.827293 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.831842 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:58.831858 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:58.831876 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:58.831915 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:58.835023 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.835761 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.835768 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.835770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.835905 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.836318 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.836322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.836324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.836697 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:58.837024 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.837330 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.837585 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":270,"threads_started":1}
W20260810 19:31:58.838704 32321 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58306
W20260810 19:31:58.839411 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32177: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:31:58.839599 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 32177
I20260810 19:31:58.867338 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32332
Aug 10 19:31:58 dist-test-slave-tm1g krb5kdc[31884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:31:58.886914 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:31:58.868291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.29.233.131:44933)
0810 19:31:58.868520 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 19:31:58.868526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:31:58.868554 (+ 28us) server_negotiation.cc:407] Connection header received
0810 19:31:58.868658 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:31:58.868664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:31:58.868759 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:31:58.868873 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:31:58.870151 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.871985 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.873385 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:31:58.874129 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:31:58.874269 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:31:58.879155 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:31:58.879180 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:31:58.879202 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:31:58.879248 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:31:58.883443 (+ 4195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.884193 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.884203 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.884207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.884396 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:31:58.884807 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:31:58.884813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:31:58.884819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:31:58.885375 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:31:58.885743 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 19:31:58.886023 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 19:31:58.886256 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 19:31:58.887742 32478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49842
W20260810 19:31:58.888513 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32332: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:31:58.888685 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 32332
I20260810 19:31:58.916949 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 31898
I20260810 19:31:58.918185 31979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:31:59.034912 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 31898
2026-08-10T19:31:59Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4967 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:31:59 dist-test-slave-tm1g krb5kdc[32585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:31:59 dist-test-slave-tm1g krb5kdc[32585](info): set up 2 sockets
Aug 10 19:31:59 dist-test-slave-tm1g krb5kdc[32585](info): commencing operation
krb5kdc: starting...
W20260810 19:32:00.198585 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:00 dist-test-slave-tm1g krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:00.430953 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:45115
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:45115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:00.763770 32603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:00.764153 32603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:00.764329 32603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:00.771744 32603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:00.771939 32603 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 19:32:00.772239 32603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:00.772389 32603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:00.772483 32603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:00.772598 32603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:00.782855 32603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:45115
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:45115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:00.784238 32603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:00.785642 32603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:00.797771 32609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:00.797919 32608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:00.797952 32612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:00.798920 32603 server_base.cc:1034] running on GCE node
I20260810 19:32:00.799693 32603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:00.801081 32603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:00.802423 32603 hybrid_clock.cc:648] HybridClock initialized: now 1786390320802289 us; error 31 us; skew 500 ppm
I20260810 19:32:00.802770 32603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:00 dist-test-slave-tm1g krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390320, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:00.806116 32603 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:00.807873 32603 webserver.cc:460] Webserver started at http://127.29.233.190:35647/ using document root <none> and password file <none>
I20260810 19:32:00.808642 32603 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:00.808776 32603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:00.809124 32603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:00.812217 32603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "04cca5fe45e94708a84aff8a35f3d8f8"
format_stamp: "Formatted at 2026-08-10 19:32:00 on dist-test-slave-tm1g"
server_key: "64023c8e3e5e02e9b2d85994e07559ae"
server_key_iv: "794e3d3b50a82da62b6fae1524fe7c5a"
server_key_version: "encryptionkey@0"
I20260810 19:32:00.813074 32603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "04cca5fe45e94708a84aff8a35f3d8f8"
format_stamp: "Formatted at 2026-08-10 19:32:00 on dist-test-slave-tm1g"
server_key: "64023c8e3e5e02e9b2d85994e07559ae"
server_key_iv: "794e3d3b50a82da62b6fae1524fe7c5a"
server_key_version: "encryptionkey@0"
I20260810 19:32:00.818804 32603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:32:00.822657 32620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:00.824148 32603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:00.824349 32603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "04cca5fe45e94708a84aff8a35f3d8f8"
format_stamp: "Formatted at 2026-08-10 19:32:00 on dist-test-slave-tm1g"
server_key: "64023c8e3e5e02e9b2d85994e07559ae"
server_key_iv: "794e3d3b50a82da62b6fae1524fe7c5a"
server_key_version: "encryptionkey@0"
I20260810 19:32:00.824632 32603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:00.838899 32603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:00.843047 32603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:00.843339 32603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:00.866755 32603 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:45115
I20260810 19:32:00.866775 32688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:45115 every 8 connection(s)
I20260810 19:32:00.868184 32603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:00.871996 32689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:00.873286 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 32603
I20260810 19:32:00.873502 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:00.874403 30630 external_mini_cluster.cc:1428] Setting key 4e2816a4147428c398f273beca5f7384
I20260810 19:32:00.885929 32689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Bootstrap starting.
Aug 10 19:32:00 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:00.889642 32689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:00.891023 32689 log.cc:824] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:00.894392 32689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: No bootstrap required, opened a new log
I20260810 19:32:00.897882 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:00.876524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.29.233.190:45115)
0810 19:32:00.877407 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:00.877419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:00.877466 (+ 47us) server_negotiation.cc:407] Connection header received
0810 19:32:00.879152 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:00.879178 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:00.879587 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:00.880006 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:00.881741 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:00.883223 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:00.884536 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:00.885221 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:00.885443 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:00.890101 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:00.890110 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:00.890129 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:00.890174 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:00.893544 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:00.894494 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:00.894505 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:00.894514 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:00.894702 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:00.895135 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:00.895138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:00.895140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:00.895653 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:00.895873 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:00.896371 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 19:32:00.896752 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":440,"threads_started":1}
I20260810 19:32:00.903378 32689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45115 } }
I20260810 19:32:00.903797 32689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:00.903911 32689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cca5fe45e94708a84aff8a35f3d8f8, State: Initialized, Role: FOLLOWER
I20260810 19:32:00.904531 32689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45115 } }
I20260810 19:32:00.904775 32689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:00.904919 32689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:00.905085 32689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:00.907606 32689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45115 } }
I20260810 19:32:00.908164 32689 leader_election.cc:302] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04cca5fe45e94708a84aff8a35f3d8f8; no voters:
I20260810 19:32:00.908830 32689 leader_election.cc:288] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:00.909004 32694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:00.909776 32694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [term 1 LEADER]: Becoming Leader. State: Replica: 04cca5fe45e94708a84aff8a35f3d8f8, State: Running, Role: LEADER
I20260810 19:32:00.910429 32694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45115 } }
I20260810 19:32:00.911051 32689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:00.913870 32696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45115 } } }
I20260810 19:32:00.914268 32696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:00.913941 32698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04cca5fe45e94708a84aff8a35f3d8f8. Latest consensus state: current_term: 1 leader_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cca5fe45e94708a84aff8a35f3d8f8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45115 } } }
I20260810 19:32:00.914722 32698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:00.917418 32702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:00.923259 32702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:00.934033 32702 catalog_manager.cc:1347] Generated new cluster ID: 921830d403bc454abc4c00466faa5b96
I20260810 19:32:00.934233 32702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:00.949312 32702 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:00.950819 32702 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:00.963722 32702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Generated new TSK 0
I20260810 19:32:00.964666 32702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:32:01.030830 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45115
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:01.378093 32721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:01.378434 32721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:01.378556 32721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:01.386891 32721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:01.387032 32721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:01.387184 32721 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 19:32:01.387349 32721 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:32:01.387496 32721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:01.397835 32721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:45115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:01.399294 32721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:01.400681 32721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:01.414288 32726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:01.414259 32728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:01.414404 32730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:01.415293 32721 server_base.cc:1034] running on GCE node
I20260810 19:32:01.415962 32721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:01.416963 32721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:01.418187 32721 hybrid_clock.cc:648] HybridClock initialized: now 1786390321418127 us; error 55 us; skew 500 ppm
I20260810 19:32:01.418542 32721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:01 dist-test-slave-tm1g krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390321, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:01.421631 32721 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:32:01.423143 32721 webserver.cc:460] Webserver started at http://127.29.233.129:45867/ using document root <none> and password file <none>
I20260810 19:32:01.423840 32721 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:01.423991 32721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:01.424312 32721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:01.427352 32721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b"
format_stamp: "Formatted at 2026-08-10 19:32:01 on dist-test-slave-tm1g"
server_key: "d751a45a15656cd62ae272c12b1626de"
server_key_iv: "de5cfc20f3d40ce4bef612e9b939c5c2"
server_key_version: "encryptionkey@0"
I20260810 19:32:01.428133 32721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b"
format_stamp: "Formatted at 2026-08-10 19:32:01 on dist-test-slave-tm1g"
server_key: "d751a45a15656cd62ae272c12b1626de"
server_key_iv: "de5cfc20f3d40ce4bef612e9b939c5c2"
server_key_version: "encryptionkey@0"
I20260810 19:32:01.433674 32721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:32:01.437494 32737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:01.439016 32721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 19:32:01.439194 32721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b"
format_stamp: "Formatted at 2026-08-10 19:32:01 on dist-test-slave-tm1g"
server_key: "d751a45a15656cd62ae272c12b1626de"
server_key_iv: "de5cfc20f3d40ce4bef612e9b939c5c2"
server_key_version: "encryptionkey@0"
I20260810 19:32:01.439400 32721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:01.460561 32721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:01.465080 32721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:01.465391 32721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:01.468165 32721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:01.468302 32721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:01.468448 32721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:01.468583 32721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 10 19:32:01 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390321, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:01.494412 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:01.468707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:44495 (local address 127.29.233.190:45115)
0810 19:32:01.471585 (+ 2878us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:01.471589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:01.471609 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:01.474073 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:01.474082 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:01.474190 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:01.474361 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:01.477115 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.478111 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.481602 (+ 3491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.482046 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.482159 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:01.487555 (+ 5396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:01.487564 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:01.487569 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:01.487611 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:01.491191 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:01.491975 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:01.491981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:01.491983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:01.492105 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:01.492566 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:01.492571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:01.492579 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:01.492847 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:01.493013 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:01.493771 (+ 758us) server_negotiation.cc:299] Negotiation successful
0810 19:32:01.494013 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2761,"thread_start_us":2665,"threads_started":1}
I20260810 19:32:01.496029 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:01.469514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45115 (local address 127.29.233.129:44495)
0810 19:32:01.470648 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:01.470707 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:01.473636 (+ 2929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:01.474673 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:01.474690 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:01.475742 (+ 1052us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:01.476874 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:01.476909 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.478336 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.478345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:01.481402 (+ 3057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:01.481424 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.482360 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.482371 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:01.482759 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:01.483721 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:01.483769 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:01.487179 (+ 3410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:01.491349 (+ 4170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:01.491358 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:01.491383 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:01.491822 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:01.492219 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:01.492224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:01.492228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:01.492432 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:01.492981 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:01.492989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:01.493380 (+ 391us) client_negotiation.cc:769] Sending connection context
0810 19:32:01.493663 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 19:32:01.494213 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":444,"threads_started":1}
I20260810 19:32:01.523612 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:01.509364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45115 (local address 127.29.233.129:42189)
0810 19:32:01.510204 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:01.510229 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:01.510422 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:01.510845 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:01.510849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:01.511347 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:01.511700 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:01.511711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.513381 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.513391 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:01.514505 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:01.514518 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.515337 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.515345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:01.515517 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:01.516248 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:01.516266 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:01.517649 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:01.520922 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:01.520928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:01.520932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:01.521344 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:01.521731 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:01.521736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:01.521740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:01.521875 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:01.522469 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:01.522479 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:01.522699 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 19:32:01.522884 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 19:32:01.523247 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":501,"threads_started":1}
I20260810 19:32:01.523878 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:01.509366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:42189 (local address 127.29.233.190:45115)
0810 19:32:01.509620 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:01.509630 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:01.510343 (+ 713us) server_negotiation.cc:407] Connection header received
0810 19:32:01.510558 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:01.510567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:01.510677 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:01.510830 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:01.511851 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.513233 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.514692 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:01.515126 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:01.515266 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:01.517870 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:01.517875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:01.517879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:01.517913 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:01.520785 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:01.521462 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:01.521477 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:01.521481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:01.521615 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:01.521982 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:01.521990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:01.521995 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:01.522290 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:01.522539 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:01.523037 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 19:32:01.523362 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 19:32:01.529018 32721 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:34225
I20260810 19:32:01.529105 400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:34225 every 8 connection(s)
I20260810 19:32:01.530391 32721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:01.532572 401 heartbeater.cc:347] Connected to a master server at 127.29.233.190:45115
I20260810 19:32:01.532832 401 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:01.533506 401 heartbeater.cc:511] Master 127.29.233.190:45115 requested a full tablet report, sending...
I20260810 19:32:01.535571 32639 ts_manager.cc:194] Registered new tserver with Master: cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225)
I20260810 19:32:01.537089 32639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:44495
I20260810 19:32:01.539815 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 32721
I20260810 19:32:01.540043 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:01.540705 30630 external_mini_cluster.cc:1428] Setting key fd7b8e703f4f46fc00c858eb013c0cf4
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:32:01.590436 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45115
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:01.940867 405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:01.941200 405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:01.941383 405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:01.948909 405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:01.949074 405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:01.949219 405 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 19:32:01.949311 405 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:32:01.949448 405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:01.959271 405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:45115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:01.960711 405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:01.962059 405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:01.975502 412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:01.975519 414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:01.975590 411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:01.976338 405 server_base.cc:1034] running on GCE node
I20260810 19:32:01.976962 405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:01.977917 405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:01.979147 405 hybrid_clock.cc:648] HybridClock initialized: now 1786390321979110 us; error 50 us; skew 500 ppm
I20260810 19:32:01.979480 405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:01 dist-test-slave-tm1g krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390321, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:01.982793 405 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:32:01.984431 405 webserver.cc:460] Webserver started at http://127.29.233.130:33191/ using document root <none> and password file <none>
I20260810 19:32:01.985136 405 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:01.985297 405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:01.985635 405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:01.988677 405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "0c931e243575460cb23c7a5db97410b7"
format_stamp: "Formatted at 2026-08-10 19:32:01 on dist-test-slave-tm1g"
server_key: "21b00df629334157a45f3ef13dc31c7f"
server_key_iv: "274aa96b8e93da9b795d9db250d15af2"
server_key_version: "encryptionkey@0"
I20260810 19:32:01.989460 405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "0c931e243575460cb23c7a5db97410b7"
format_stamp: "Formatted at 2026-08-10 19:32:01 on dist-test-slave-tm1g"
server_key: "21b00df629334157a45f3ef13dc31c7f"
server_key_iv: "274aa96b8e93da9b795d9db250d15af2"
server_key_version: "encryptionkey@0"
I20260810 19:32:01.994903 405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 19:32:01.998533 423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:02.000005 405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:02.000137 405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "0c931e243575460cb23c7a5db97410b7"
format_stamp: "Formatted at 2026-08-10 19:32:01 on dist-test-slave-tm1g"
server_key: "21b00df629334157a45f3ef13dc31c7f"
server_key_iv: "274aa96b8e93da9b795d9db250d15af2"
server_key_version: "encryptionkey@0"
I20260810 19:32:02.000305 405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:02.022959 405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:02.027256 405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:02.027530 405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:02.030645 405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:02.030771 405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:02.030910 405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:02.031010 405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:32:02 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390321, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:02.056414 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.031271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:34145 (local address 127.29.233.190:45115)
0810 19:32:02.031997 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.032004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.033033 (+ 1029us) server_negotiation.cc:407] Connection header received
0810 19:32:02.035339 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.035345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.035436 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.035572 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:02.037899 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.039296 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.042051 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.042804 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.042952 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.048485 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:02.048493 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:02.048501 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:02.048546 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:02.052236 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.053374 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.053390 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.053396 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.053586 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.054140 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.054148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.054153 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.054627 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:02.054799 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.055769 (+ 970us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.056011 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":467,"threads_started":1}
I20260810 19:32:02.058012 434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.032007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45115 (local address 127.29.233.130:34145)
0810 19:32:02.032852 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:02.032899 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:02.035016 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:02.035866 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:02.035881 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:02.036765 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:02.037663 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.037704 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.039503 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.039516 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.041808 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.041832 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.043129 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.043140 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.043447 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.044378 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:02.044428 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:02.048114 (+ 3686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:02.052401 (+ 4287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.052412 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.052444 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.053193 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.053750 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.053754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.053760 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.053999 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.054777 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:02.054789 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:02.055276 (+ 487us) client_negotiation.cc:769] Sending connection context
0810 19:32:02.055674 (+ 398us) client_negotiation.cc:240] Negotiation successful
0810 19:32:02.056312 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":390,"threads_started":1}
I20260810 19:32:02.073820 405 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:44053
I20260810 19:32:02.073873 557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:44053 every 8 connection(s)
I20260810 19:32:02.075181 405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:02.077500 558 heartbeater.cc:347] Connected to a master server at 127.29.233.190:45115
I20260810 19:32:02.077764 558 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:02.078708 558 heartbeater.cc:511] Master 127.29.233.190:45115 requested a full tablet report, sending...
I20260810 19:32:02.080355 32639 ts_manager.cc:194] Registered new tserver with Master: 0c931e243575460cb23c7a5db97410b7 (127.29.233.130:44053)
I20260810 19:32:02.081620 32639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:34145
I20260810 19:32:02.083031 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 405
I20260810 19:32:02.083223 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:02.083832 30630 external_mini_cluster.cc:1428] Setting key 0b9a27dc03196b7d8e7514db17e93655
I20260810 19:32:02.086546 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.069791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45115 (local address 127.29.233.130:35883)
0810 19:32:02.071259 (+ 1468us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:02.071278 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:02.071537 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:02.072012 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:02.072018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:02.072836 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:02.073485 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.073506 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.075030 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.075036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.076399 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.076418 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.077737 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.077746 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.077953 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.078662 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:02.078690 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:02.079750 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:02.083773 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.083777 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.083781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.084309 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.084732 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.084736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.084738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.084926 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.085553 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:02.085559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:02.085782 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 19:32:02.085924 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 19:32:02.086145 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"thread_start_us":1028,"threads_started":1}
I20260810 19:32:02.086854 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.070329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:35883 (local address 127.29.233.190:45115)
0810 19:32:02.070571 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.070580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.071392 (+ 812us) server_negotiation.cc:407] Connection header received
0810 19:32:02.071730 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.071736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.071829 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.071958 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:02.073789 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.074882 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.076652 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.077299 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.077408 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.079966 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:02.079972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:02.079976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:02.080022 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:02.083637 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.084434 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.084441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.084444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.084613 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.085041 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.085045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.085047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.085422 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:02.085569 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.086085 (+ 516us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.086348 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:32:02.134232 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45115
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:02.475805 562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:02.476154 562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:02.476343 562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:02.483598 562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:02.483765 562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:02.483932 562 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 19:32:02.484025 562 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:32:02.484155 562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:02.494129 562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:45115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:02.495651 562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:02.497081 562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:02.510955 567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:02.511021 568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:02.510998 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:02.512264 562 server_base.cc:1034] running on GCE node
I20260810 19:32:02.512910 562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:02.513837 562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:02.515049 562 hybrid_clock.cc:648] HybridClock initialized: now 1786390322515010 us; error 30 us; skew 500 ppm
I20260810 19:32:02.515343 562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:02 dist-test-slave-tm1g krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390322, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:02.518980 562 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:32:02.520753 562 webserver.cc:460] Webserver started at http://127.29.233.131:46777/ using document root <none> and password file <none>
I20260810 19:32:02.521552 562 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:02.521705 562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:02.522082 562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:02.525461 562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "270318e72d2345c89d43d493ff66eec3"
format_stamp: "Formatted at 2026-08-10 19:32:02 on dist-test-slave-tm1g"
server_key: "c2cf3fa7cd2f0f1b6d990592e3d2e108"
server_key_iv: "1ffccc2228d0479dbf46fd7aa238d125"
server_key_version: "encryptionkey@0"
I20260810 19:32:02.526419 562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "270318e72d2345c89d43d493ff66eec3"
format_stamp: "Formatted at 2026-08-10 19:32:02 on dist-test-slave-tm1g"
server_key: "c2cf3fa7cd2f0f1b6d990592e3d2e108"
server_key_iv: "1ffccc2228d0479dbf46fd7aa238d125"
server_key_version: "encryptionkey@0"
I20260810 19:32:02.532742 562 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 19:32:02.536290 579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:02.537603 562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:02.537781 562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "270318e72d2345c89d43d493ff66eec3"
format_stamp: "Formatted at 2026-08-10 19:32:02 on dist-test-slave-tm1g"
server_key: "c2cf3fa7cd2f0f1b6d990592e3d2e108"
server_key_iv: "1ffccc2228d0479dbf46fd7aa238d125"
server_key_version: "encryptionkey@0"
I20260810 19:32:02.537971 562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:02.539921 401 heartbeater.cc:503] Master 127.29.233.190:45115 was elected leader, sending a full tablet report...
I20260810 19:32:02.561317 562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:02.565572 562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:02.565821 562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:02.568538 562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:02.568663 562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:02.568789 562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:02.568881 562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:32:02 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390322, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:02.598204 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.569205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:32981 (local address 127.29.233.190:45115)
0810 19:32:02.569428 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.569437 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.570953 (+ 1516us) server_negotiation.cc:407] Connection header received
0810 19:32:02.573032 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.573038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.573136 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.573254 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:02.575477 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.576571 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.579450 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.580177 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.580335 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.585704 (+ 5369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:02.585714 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:02.585721 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:02.585772 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:02.594170 (+ 8398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.595285 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.595293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.595299 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.595501 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.596008 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.596014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.596017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.596351 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:02.596542 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.597719 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.597897 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 19:32:02.600344 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.569903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45115 (local address 127.29.233.131:32981)
0810 19:32:02.570789 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:02.570833 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:02.572716 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:02.573499 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:02.573510 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:02.574259 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:02.575264 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.575296 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.576745 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.576749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.579269 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.579288 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.580439 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.580446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.580628 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.581420 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:02.581464 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:02.585411 (+ 3947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:02.594351 (+ 8940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.594361 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.594384 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.595083 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.595655 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.595662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.595667 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.595873 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.596533 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:02.596546 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:02.597192 (+ 646us) client_negotiation.cc:769] Sending connection context
0810 19:32:02.597590 (+ 398us) client_negotiation.cc:240] Negotiation successful
0810 19:32:02.598379 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":415,"threads_started":1}
I20260810 19:32:02.616664 562 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:39341
I20260810 19:32:02.616683 708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:39341 every 8 connection(s)
I20260810 19:32:02.618362 562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:02.620663 709 heartbeater.cc:347] Connected to a master server at 127.29.233.190:45115
I20260810 19:32:02.620903 709 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:02.621589 709 heartbeater.cc:511] Master 127.29.233.190:45115 requested a full tablet report, sending...
I20260810 19:32:02.623008 32639 ts_manager.cc:194] Registered new tserver with Master: 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:02.623413 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.607549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:35245 (local address 127.29.233.190:45115)
0810 19:32:02.607741 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.607750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.608697 (+ 947us) server_negotiation.cc:407] Connection header received
0810 19:32:02.608886 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.608891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.608972 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.609074 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:02.610008 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.611153 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.613691 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.614157 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.614251 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.616874 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:02.616882 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:02.616886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:02.616930 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:02.620481 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.621283 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.621290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.621294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.621472 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.621891 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.621897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.621901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.622174 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:02.622346 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.622761 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.622998 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 19:32:02.623446 699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.607911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45115 (local address 127.29.233.131:35245)
0810 19:32:02.608592 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:02.608611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:02.608767 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:02.609099 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:02.609104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:02.609577 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:02.609864 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.609873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.611309 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.611319 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.613515 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.613538 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.614397 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.614407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.614630 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.615261 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:02.615284 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:02.616643 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:02.620630 (+ 3987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.620636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.620640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.621163 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.621594 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:02.621599 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:02.621603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:02.621776 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:02.622293 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:02.622298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:02.622520 (+ 222us) client_negotiation.cc:769] Sending connection context
0810 19:32:02.622663 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 19:32:02.622992 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":414,"threads_started":1}
I20260810 19:32:02.624046 32639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:32981
I20260810 19:32:02.628191 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 562
I20260810 19:32:02.628372 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:02.628878 30630 external_mini_cluster.cc:1428] Setting key e8e5158de705253147b32fb8c9f8cb22
I20260810 19:32:02.631810 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 19:32:02 dist-test-slave-tm1g krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 19:32:02 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:02.686947 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.672766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.29.233.190:45115)
0810 19:32:02.673016 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.673024 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.673345 (+ 321us) server_negotiation.cc:407] Connection header received
0810 19:32:02.673570 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.673577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.673648 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.673757 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:02.675324 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.676080 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.677299 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.677728 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.677864 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.681690 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:02.681695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:02.681698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:02.681731 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:02.684285 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.684906 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.684913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.684916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.685044 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:02.685437 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:02.685468 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:02.685470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:02.685743 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:02.686016 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.686372 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.686621 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 19:32:02.690992 32639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:02.697989 32639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:02.735780 725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.711900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:44053 (local address 127.0.0.1:45794)
0810 19:32:02.723796 (+ 11896us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:02.723819 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:02.723953 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:02.726110 (+ 2157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:02.726122 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:02.726186 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:02.726768 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.726785 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.729382 (+ 2597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.729391 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.732543 (+ 3152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.732560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.734739 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.734746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.734979 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.735041 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 19:32:02.735175 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 19:32:02.735357 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11710,"thread_start_us":12125,"threads_started":1}
I20260810 19:32:02.735852 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.712396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.29.233.130:44053)
0810 19:32:02.725173 (+ 12777us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.725187 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.725232 (+ 45us) server_negotiation.cc:407] Connection header received
0810 19:32:02.725400 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.725414 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.725772 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.726044 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:02.726984 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.729089 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.732763 (+ 3674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.734524 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.734729 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.734926 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.735285 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.735492 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12529,"thread_start_us":13900,"threads_started":1}
I20260810 19:32:02.738009 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.715006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:34225 (local address 127.0.0.1:60180)
0810 19:32:02.730718 (+ 15712us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:02.730746 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:02.730890 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:02.731451 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:02.731456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:02.731482 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:02.731883 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.731893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.733573 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.733579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.735545 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.735560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.737159 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.737165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.737341 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.737374 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 19:32:02.737456 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 19:32:02.737586 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15433,"mutex_wait_us":1,"thread_start_us":6464,"threads_started":1}
I20260810 19:32:02.739094 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.711518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:39341 (local address 127.0.0.1:52900)
0810 19:32:02.721940 (+ 10422us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:02.721993 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:02.722193 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:02.729683 (+ 7490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:02.729689 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:02.729720 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:02.731043 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.731054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.733290 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.733296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.735820 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:02.735840 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.738214 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.738221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:02.738436 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.738474 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 19:32:02.738577 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 19:32:02.738733 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10156,"thread_start_us":18664,"threads_started":1}
I20260810 19:32:02.739109 730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.715320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.29.233.129:34225)
0810 19:32:02.730648 (+ 15328us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.730659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.730841 (+ 182us) server_negotiation.cc:407] Connection header received
0810 19:32:02.731072 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.731082 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.731288 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.731470 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:02.732037 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.733417 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.735736 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.736951 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.738289 (+ 1338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.738523 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.738629 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.738800 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15164,"thread_start_us":15137,"threads_started":1}
I20260810 19:32:02.739136 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:02.712061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.29.233.131:39341)
0810 19:32:02.729093 (+ 17032us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:02.729103 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:02.729138 (+ 35us) server_negotiation.cc:407] Connection header received
0810 19:32:02.729239 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:02.729246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:02.729499 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:02.729746 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:02.731305 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.733107 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.736053 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:02.737614 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:02.737829 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:02.738012 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:02.738648 (+ 636us) server_negotiation.cc:299] Negotiation successful
0810 19:32:02.738807 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16826,"thread_start_us":16549,"threads_started":1}
I20260810 19:32:02.740868 481 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc5bc910b094bc88fd5ca594c027f52 (DEFAULT_TABLE table=test-table [id=15f8dd69563445dda80cf2d2bf36d38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:02.742663 481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc5bc910b094bc88fd5ca594c027f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:02.742431 633 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc5bc910b094bc88fd5ca594c027f52 (DEFAULT_TABLE table=test-table [id=15f8dd69563445dda80cf2d2bf36d38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:02.743046 325 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc5bc910b094bc88fd5ca594c027f52 (DEFAULT_TABLE table=test-table [id=15f8dd69563445dda80cf2d2bf36d38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:02.743850 633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc5bc910b094bc88fd5ca594c027f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:02.744535 325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc5bc910b094bc88fd5ca594c027f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:02.755564 731 tablet_bootstrap.cc:492] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Bootstrap starting.
I20260810 19:32:02.759063 731 tablet_bootstrap.cc:654] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:02.759152 732 tablet_bootstrap.cc:492] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Bootstrap starting.
I20260810 19:32:02.760546 731 log.cc:824] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:02.762928 732 tablet_bootstrap.cc:654] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:02.764166 731 tablet_bootstrap.cc:492] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: No bootstrap required, opened a new log
I20260810 19:32:02.764463 731 ts_tablet_manager.cc:1397] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 19:32:02.764633 732 log.cc:824] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:02.762109 733 tablet_bootstrap.cc:492] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Bootstrap starting.
I20260810 19:32:02.767627 732 tablet_bootstrap.cc:492] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: No bootstrap required, opened a new log
I20260810 19:32:02.767892 732 ts_tablet_manager.cc:1397] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 19:32:02.769654 733 tablet_bootstrap.cc:654] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:02.770941 733 log.cc:824] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:02.773775 733 tablet_bootstrap.cc:492] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: No bootstrap required, opened a new log
I20260810 19:32:02.774075 733 ts_tablet_manager.cc:1397] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 19:32:02.775619 731 raft_consensus.cc:348] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:02.776108 731 raft_consensus.cc:374] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:02.776216 731 raft_consensus.cc:729] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270318e72d2345c89d43d493ff66eec3, State: Initialized, Role: FOLLOWER
I20260810 19:32:02.776841 731 consensus_queue.cc:260] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:02.778033 709 heartbeater.cc:503] Master 127.29.233.190:45115 was elected leader, sending a full tablet report...
I20260810 19:32:02.778821 731 ts_tablet_manager.cc:1428] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260810 19:32:02.779219 732 raft_consensus.cc:348] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:02.779687 732 raft_consensus.cc:374] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:02.779814 732 raft_consensus.cc:729] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Initialized, Role: FOLLOWER
I20260810 19:32:02.780404 732 consensus_queue.cc:260] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:02.781405 558 heartbeater.cc:503] Master 127.29.233.190:45115 was elected leader, sending a full tablet report...
I20260810 19:32:02.783418 732 ts_tablet_manager.cc:1428] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260810 19:32:02.783774 733 raft_consensus.cc:348] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:02.784245 733 raft_consensus.cc:374] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:02.784387 733 raft_consensus.cc:729] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd8971ae77b41e1aa9a26d7f0cd6a9b, State: Initialized, Role: FOLLOWER
I20260810 19:32:02.785051 733 consensus_queue.cc:260] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:02.786689 733 ts_tablet_manager.cc:1428] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260810 19:32:02.827432 559 tablet.cc:2369] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:32:02.870607 710 tablet.cc:2369] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:32:03.033011 402 tablet.cc:2369] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:32:03.049537 738 raft_consensus.cc:482] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:03.049908 738 raft_consensus.cc:504] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:03.051707 738 leader_election.cc:288] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341), cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225)
I20260810 19:32:03.059024 730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.052290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:58519 (local address 127.29.233.129:34225)
0810 19:32:03.052520 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:03.052538 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:03.053349 (+ 811us) server_negotiation.cc:407] Connection header received
0810 19:32:03.053518 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:03.053524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:03.053624 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:03.053707 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:03.054569 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.055348 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.057269 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.058020 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.058196 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.058283 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:03.058617 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 19:32:03.058745 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 19:32:03.059054 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.052280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:40033 (local address 127.29.233.131:39341)
0810 19:32:03.052515 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:03.052525 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:03.053479 (+ 954us) server_negotiation.cc:407] Connection header received
0810 19:32:03.053781 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:03.053787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:03.053870 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:03.053988 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:03.054711 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.055443 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.057274 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.058025 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.058218 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.058330 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:03.058650 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 19:32:03.058772 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 19:32:03.059190 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.052274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:34225 (local address 127.29.233.130:58519)
0810 19:32:03.053208 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:03.053225 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:03.053375 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:03.053780 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:03.053785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:03.053823 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:03.054404 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.054420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.055482 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.055490 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.057102 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.057115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.058250 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.058259 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.058446 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.058481 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 19:32:03.058559 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 19:32:03.058709 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":68,"thread_start_us":527,"threads_started":1}
I20260810 19:32:03.059238 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.052292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:39341 (local address 127.29.233.130:40033)
0810 19:32:03.053350 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:03.053371 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:03.053516 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:03.054017 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:03.054020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:03.054040 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:03.054550 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.054563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.055589 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.055593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.057104 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.057116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.058276 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.058282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.058473 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.058496 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:32:03.058607 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 19:32:03.058740 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":55,"thread_start_us":548,"threads_started":1}
I20260810 19:32:03.060402 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc5bc910b094bc88fd5ca594c027f52" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" is_pre_election: true
I20260810 19:32:03.060413 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc5bc910b094bc88fd5ca594c027f52" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3" is_pre_election: true
I20260810 19:32:03.061020 656 raft_consensus.cc:2454] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:03.061015 347 raft_consensus.cc:2454] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:03.061717 424 leader_election.cc:302] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, cfd8971ae77b41e1aa9a26d7f0cd6a9b; no voters:
I20260810 19:32:03.062100 738 raft_consensus.cc:2781] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:03.062264 738 raft_consensus.cc:482] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:03.062412 738 raft_consensus.cc:3037] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.064561 738 raft_consensus.cc:504] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:03.065158 738 leader_election.cc:288] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Requested vote from peers 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341), cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225)
I20260810 19:32:03.065646 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc5bc910b094bc88fd5ca594c027f52" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3"
I20260810 19:32:03.065680 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc5bc910b094bc88fd5ca594c027f52" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b"
I20260810 19:32:03.065846 656 raft_consensus.cc:3037] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.065898 347 raft_consensus.cc:3037] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.068205 347 raft_consensus.cc:2454] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:03.068321 656 raft_consensus.cc:2454] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:03.068641 424 leader_election.cc:302] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, cfd8971ae77b41e1aa9a26d7f0cd6a9b; no voters:
I20260810 19:32:03.068953 738 raft_consensus.cc:2781] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:03.069391 738 raft_consensus.cc:686] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 1 LEADER]: Becoming Leader. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Running, Role: LEADER
I20260810 19:32:03.069911 738 consensus_queue.cc:237] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } }
I20260810 19:32:03.075104 32638 catalog_manager.cc:5495] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c931e243575460cb23c7a5db97410b7 (127.29.233.130). New cstate: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } health_report { overall_health: UNKNOWN } } }
Aug 10 19:32:03 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390320, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:03.135002 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.121056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45115 (local address 127.0.0.1:43064)
0810 19:32:03.121620 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:03.121638 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:03.121822 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:03.122206 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:03.122211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:03.122982 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:03.123277 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.123287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.124353 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.124361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.125366 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.125378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.126125 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.126133 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.126328 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.127011 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:03.127034 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:03.129481 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:03.132570 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:03.132581 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:03.132586 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:03.133044 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:03.133382 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:03.133385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:03.133387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:03.133546 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:03.134014 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:03.134020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:03.134167 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 19:32:03.134374 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 19:32:03.134615 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":306,"threads_started":1}
I20260810 19:32:03.135025 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.120971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.29.233.190:45115)
0810 19:32:03.121199 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:03.121203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:03.121737 (+ 534us) server_negotiation.cc:407] Connection header received
0810 19:32:03.121977 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:03.121982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:03.122064 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:03.122174 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:03.123436 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.124220 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.125513 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.125928 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.126058 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.129661 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:03.129666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:03.129669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:03.129699 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:03.132423 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:03.133151 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:03.133155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:03.133157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:03.133284 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:03.133637 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:03.133641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:03.133643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:03.133892 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:03.134031 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:03.134477 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 19:32:03.134737 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":81}
I20260810 19:32:03.140221 32638 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:43064:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 19:32:03.141206 32638 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:03.148087 325 tablet_service.cc:1467] Processing CreateTablet for tablet e1801eade28841b18ef56ec2bc37de69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87f7ea6647654d7ead8633532ca53800]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:32:03.148125 481 tablet_service.cc:1467] Processing CreateTablet for tablet e1801eade28841b18ef56ec2bc37de69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87f7ea6647654d7ead8633532ca53800]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:32:03.148453 633 tablet_service.cc:1467] Processing CreateTablet for tablet e1801eade28841b18ef56ec2bc37de69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87f7ea6647654d7ead8633532ca53800]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:32:03.149138 481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1801eade28841b18ef56ec2bc37de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:03.149138 325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1801eade28841b18ef56ec2bc37de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:03.149207 633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1801eade28841b18ef56ec2bc37de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:03.155030 731 tablet_bootstrap.cc:492] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: Bootstrap starting.
I20260810 19:32:03.156396 732 tablet_bootstrap.cc:492] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: Bootstrap starting.
I20260810 19:32:03.156553 731 tablet_bootstrap.cc:654] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:03.157802 733 tablet_bootstrap.cc:492] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Bootstrap starting.
I20260810 19:32:03.158133 732 tablet_bootstrap.cc:654] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:03.159062 731 tablet_bootstrap.cc:492] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: No bootstrap required, opened a new log
I20260810 19:32:03.159230 731 ts_tablet_manager.cc:1397] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 19:32:03.159284 733 tablet_bootstrap.cc:654] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:03.159832 731 raft_consensus.cc:348] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.160089 731 raft_consensus.cc:374] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:03.160182 731 raft_consensus.cc:729] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270318e72d2345c89d43d493ff66eec3, State: Initialized, Role: FOLLOWER
I20260810 19:32:03.160552 732 tablet_bootstrap.cc:492] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: No bootstrap required, opened a new log
I20260810 19:32:03.160452 731 consensus_queue.cc:260] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.160753 732 ts_tablet_manager.cc:1397] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260810 19:32:03.161089 737 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.161406 737 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:03.161584 732 raft_consensus.cc:348] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.161767 731 ts_tablet_manager.cc:1428] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 19:32:03.161819 732 raft_consensus.cc:374] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:03.161907 732 raft_consensus.cc:729] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Initialized, Role: FOLLOWER
I20260810 19:32:03.162132 732 consensus_queue.cc:260] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.162281 733 tablet_bootstrap.cc:492] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: No bootstrap required, opened a new log
I20260810 19:32:03.162464 733 ts_tablet_manager.cc:1397] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 19:32:03.162621 738 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.162905 738 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:03.163218 733 raft_consensus.cc:348] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.163388 732 ts_tablet_manager.cc:1428] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 19:32:03.163481 733 raft_consensus.cc:374] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:03.164358 733 raft_consensus.cc:729] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd8971ae77b41e1aa9a26d7f0cd6a9b, State: Initialized, Role: FOLLOWER
I20260810 19:32:03.164614 733 consensus_queue.cc:260] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.165529 733 ts_tablet_manager.cc:1428] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 19:32:03.165777 739 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.166168 739 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:03.244666 738 raft_consensus.cc:482] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:03.244906 738 raft_consensus.cc:504] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.245604 738 leader_election.cc:288] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:03.246184 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1801eade28841b18ef56ec2bc37de69" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3" is_pre_election: true
I20260810 19:32:03.246188 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1801eade28841b18ef56ec2bc37de69" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" is_pre_election: true
I20260810 19:32:03.246526 347 raft_consensus.cc:2454] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:03.246521 656 raft_consensus.cc:2454] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:03.247018 424 leader_election.cc:302] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, cfd8971ae77b41e1aa9a26d7f0cd6a9b; no voters:
I20260810 19:32:03.247344 738 raft_consensus.cc:2781] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:03.247434 738 raft_consensus.cc:482] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:03.247597 738 raft_consensus.cc:3037] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.249768 738 raft_consensus.cc:504] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.250432 738 leader_election.cc:288] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Requested vote from peers cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:03.250874 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1801eade28841b18ef56ec2bc37de69" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3"
I20260810 19:32:03.250878 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1801eade28841b18ef56ec2bc37de69" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b"
I20260810 19:32:03.251117 347 raft_consensus.cc:3037] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.251124 656 raft_consensus.cc:3037] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.253257 656 raft_consensus.cc:2454] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:03.253681 427 leader_election.cc:302] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, 270318e72d2345c89d43d493ff66eec3; no voters:
I20260810 19:32:03.253810 347 raft_consensus.cc:2454] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:03.254076 738 raft_consensus.cc:2781] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:03.254338 738 raft_consensus.cc:686] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [term 1 LEADER]: Becoming Leader. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Running, Role: LEADER
I20260810 19:32:03.254637 738 consensus_queue.cc:237] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.256281 743 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: TxnStatusTablet state changed. Reason: New leader 0c931e243575460cb23c7a5db97410b7. Latest consensus state: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.256599 743 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:32:03.257272 756 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 19:32:03.257576 756 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 19:32:03.257647 32637 catalog_manager.cc:5495] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c931e243575460cb23c7a5db97410b7 (127.29.233.130). New cstate: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:03.291625 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.284987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.29.233.130:44053)
0810 19:32:03.285572 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:03.285576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:03.285592 (+ 16us) server_negotiation.cc:407] Connection header received
0810 19:32:03.285664 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:03.285668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:03.285737 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:03.285862 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:03.286397 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.287284 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.289180 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.289610 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.289707 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.290293 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:03.290964 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:03.291067 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:03.291241 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 19:32:03.291311 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":306,"threads_started":1}
I20260810 19:32:03.291671 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.284926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:44053 (local address 127.0.0.1:45806)
0810 19:32:03.285189 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:03.285211 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:03.285371 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:03.285945 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:03.285952 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:03.285965 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:03.286169 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.286180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.287421 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.287427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.289038 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.289055 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.289781 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.289788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.289997 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.290074 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:03.291077 (+ 1003us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:03.291102 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 19:32:03.291201 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 19:32:03.291366 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 19:32:03.299546 656 raft_consensus.cc:1261] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:03.299588 347 raft_consensus.cc:1261] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:03.300583 743 consensus_queue.cc:1035] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:03.300954 738 consensus_queue.cc:1035] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:03.318207 739 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: TxnStatusTablet state changed. Reason: New leader 0c931e243575460cb23c7a5db97410b7. Latest consensus state: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.318588 739 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:03.318838 737 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: TxnStatusTablet state changed. Reason: New leader 0c931e243575460cb23c7a5db97410b7. Latest consensus state: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.319139 737 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:03.321345 760 mvcc.cc:205] Tried to move back new op lower bound from 7317054764224299008 to 7317054764053352448. Current Snapshot: MvccSnapshot[applied={T|T < 7317054764224299008}]
I20260810 19:32:03.321895 761 mvcc.cc:205] Tried to move back new op lower bound from 7317054764224299008 to 7317054764053352448. Current Snapshot: MvccSnapshot[applied={T|T < 7317054764224299008}]
I20260810 19:32:03.326807 762 mvcc.cc:205] Tried to move back new op lower bound from 7317054764224299008 to 7317054764053352448. Current Snapshot: MvccSnapshot[applied={T|T < 7317054764224299008}]
I20260810 19:32:03.327834 743 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.328243 743 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:32:03.331295 743 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.331568 743 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P 0c931e243575460cb23c7a5db97410b7: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:32:03.335410 739 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.335664 739 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:03.336428 737 tablet_replica.cc:439] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } } }
I20260810 19:32:03.336690 737 tablet_replica.cc:442] T e1801eade28841b18ef56ec2bc37de69 P 270318e72d2345c89d43d493ff66eec3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:03.348924 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.342093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.29.233.130:44053)
0810 19:32:03.342290 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:03.342320 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:03.342913 (+ 593us) server_negotiation.cc:407] Connection header received
0810 19:32:03.343078 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:03.343084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:03.343171 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:03.343292 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:03.344043 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.344797 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.347073 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.347567 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.347681 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.348194 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:03.348303 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:03.348386 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:03.348544 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 19:32:03.348608 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 19:32:03.367448 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.360550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:42121 (local address 127.29.233.130:44053)
0810 19:32:03.360771 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:03.360777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:03.361444 (+ 667us) server_negotiation.cc:407] Connection header received
0810 19:32:03.361640 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:03.361648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:03.361753 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:03.361860 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:03.362469 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.363652 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.365131 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.365841 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.365973 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.366560 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:03.366691 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:03.366787 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:03.366964 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 19:32:03.367055 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 19:32:03.367472 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:03.360581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:44053 (local address 127.29.233.130:42121)
0810 19:32:03.361351 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:03.361367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:03.361519 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:03.361920 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:03.361924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:03.361934 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:03.362258 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.362265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.363799 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.363803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.364995 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:03.365007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:03.366054 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:03.366059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:03.366204 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:03.366265 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:03.366818 (+ 553us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:03.366829 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 19:32:03.366915 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 19:32:03.367025 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":412,"threads_started":1}
I20260810 19:32:03.377465 656 raft_consensus.cc:1261] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:03.377478 346 raft_consensus.cc:1261] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:03.378010 743 consensus_queue.cc:1035] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:03.378353 738 consensus_queue.cc:1035] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:03.449383 32638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:03.449869 32638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:03.453550 32638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Sending DeleteTablet for 3 replicas of tablet 8dc5bc910b094bc88fd5ca594c027f52
I20260810 19:32:03.454809 481 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc5bc910b094bc88fd5ca594c027f52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:03 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:45794
I20260810 19:32:03.454803 633 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc5bc910b094bc88fd5ca594c027f52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:03 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52900
I20260810 19:32:03.454803 325 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc5bc910b094bc88fd5ca594c027f52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:03 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:60180
I20260810 19:32:03.458467 32637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:03.459342 32637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:03.467767 781 tablet_replica.cc:330] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: stopping tablet replica
I20260810 19:32:03.468921 783 tablet_replica.cc:330] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: stopping tablet replica
I20260810 19:32:03.470028 633 tablet_service.cc:1467] Processing CreateTablet for tablet 43c68939cdb24224bf84d419b4664bd6 (DEFAULT_TABLE table=test-table [id=36fb0813437b4e55891ea5a5e266771d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:03.470662 324 tablet_service.cc:1467] Processing CreateTablet for tablet 43c68939cdb24224bf84d419b4664bd6 (DEFAULT_TABLE table=test-table [id=36fb0813437b4e55891ea5a5e266771d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:03.471060 633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c68939cdb24224bf84d419b4664bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:03.471091 480 tablet_service.cc:1467] Processing CreateTablet for tablet 43c68939cdb24224bf84d419b4664bd6 (DEFAULT_TABLE table=test-table [id=36fb0813437b4e55891ea5a5e266771d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:03.471243 324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c68939cdb24224bf84d419b4664bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:03.471624 480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c68939cdb24224bf84d419b4664bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:03.474570 783 raft_consensus.cc:2229] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:03.474843 781 raft_consensus.cc:2229] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:03.478628 732 tablet_bootstrap.cc:492] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: Bootstrap starting.
I20260810 19:32:03.479151 733 tablet_bootstrap.cc:492] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Bootstrap starting.
I20260810 19:32:03.479979 781 raft_consensus.cc:2258] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:03.480700 783 raft_consensus.cc:2258] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:03.481498 782 tablet_replica.cc:330] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: stopping tablet replica
I20260810 19:32:03.481896 732 tablet_bootstrap.cc:654] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:03.482041 733 tablet_bootstrap.cc:654] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:03.481942 782 raft_consensus.cc:2229] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:03.483644 783 ts_tablet_manager.cc:1905] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:03.483822 781 ts_tablet_manager.cc:1905] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:03.493652 783 ts_tablet_manager.cc:1918] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:03.493896 783 log.cc:1196] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/8dc5bc910b094bc88fd5ca594c027f52
I20260810 19:32:03.494837 783 ts_tablet_manager.cc:1939] T 8dc5bc910b094bc88fd5ca594c027f52 P 270318e72d2345c89d43d493ff66eec3: Deleting consensus metadata
I20260810 19:32:03.495229 731 tablet_bootstrap.cc:492] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: Bootstrap starting.
I20260810 19:32:03.496248 733 tablet_bootstrap.cc:492] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: No bootstrap required, opened a new log
I20260810 19:32:03.496474 733 ts_tablet_manager.cc:1397] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.012s
I20260810 19:32:03.496937 732 tablet_bootstrap.cc:492] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: No bootstrap required, opened a new log
I20260810 19:32:03.496941 32624 catalog_manager.cc:4841] TS 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341): tablet 8dc5bc910b094bc88fd5ca594c027f52 (table test-table [id=15f8dd69563445dda80cf2d2bf36d38c]) successfully deleted
I20260810 19:32:03.497128 732 ts_tablet_manager.cc:1397] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.008s
I20260810 19:32:03.497296 731 tablet_bootstrap.cc:654] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:03.497370 733 raft_consensus.cc:348] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.497639 733 raft_consensus.cc:374] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:03.497730 733 raft_consensus.cc:729] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd8971ae77b41e1aa9a26d7f0cd6a9b, State: Initialized, Role: FOLLOWER
I20260810 19:32:03.498049 733 consensus_queue.cc:260] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.499361 733 ts_tablet_manager.cc:1428] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 19:32:03.500687 781 ts_tablet_manager.cc:1918] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:03.500906 781 log.cc:1196] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/8dc5bc910b094bc88fd5ca594c027f52
I20260810 19:32:03.501227 782 raft_consensus.cc:2258] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:03.501530 781 ts_tablet_manager.cc:1939] T 8dc5bc910b094bc88fd5ca594c027f52 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting consensus metadata
I20260810 19:32:03.503953 32621 catalog_manager.cc:4841] TS cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225): tablet 8dc5bc910b094bc88fd5ca594c027f52 (table test-table [id=15f8dd69563445dda80cf2d2bf36d38c]) successfully deleted
I20260810 19:32:03.504634 782 ts_tablet_manager.cc:1905] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:03.507169 732 raft_consensus.cc:348] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.507498 732 raft_consensus.cc:374] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:03.507599 732 raft_consensus.cc:729] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Initialized, Role: FOLLOWER
I20260810 19:32:03.507911 732 consensus_queue.cc:260] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.508078 731 tablet_bootstrap.cc:492] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: No bootstrap required, opened a new log
I20260810 19:32:03.508389 731 ts_tablet_manager.cc:1397] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.001s
I20260810 19:32:03.509238 731 raft_consensus.cc:348] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.509552 731 raft_consensus.cc:374] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:03.509675 731 raft_consensus.cc:729] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270318e72d2345c89d43d493ff66eec3, State: Initialized, Role: FOLLOWER
I20260810 19:32:03.510016 731 consensus_queue.cc:260] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.511044 732 ts_tablet_manager.cc:1428] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260810 19:32:03.511814 731 ts_tablet_manager.cc:1428] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:03.513593 782 ts_tablet_manager.cc:1918] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:03.513797 782 log.cc:1196] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/8dc5bc910b094bc88fd5ca594c027f52
I20260810 19:32:03.514482 782 ts_tablet_manager.cc:1939] T 8dc5bc910b094bc88fd5ca594c027f52 P 0c931e243575460cb23c7a5db97410b7: Deleting consensus metadata
I20260810 19:32:03.516386 32622 catalog_manager.cc:4841] TS 0c931e243575460cb23c7a5db97410b7 (127.29.233.130:44053): tablet 8dc5bc910b094bc88fd5ca594c027f52 (table test-table [id=15f8dd69563445dda80cf2d2bf36d38c]) successfully deleted
I20260810 19:32:03.863595 779 raft_consensus.cc:482] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:03.863850 779 raft_consensus.cc:504] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.864639 779 leader_election.cc:288] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:03.865237 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c68939cdb24224bf84d419b4664bd6" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3" is_pre_election: true
I20260810 19:32:03.865226 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c68939cdb24224bf84d419b4664bd6" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" is_pre_election: true
I20260810 19:32:03.865527 656 raft_consensus.cc:2454] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:03.865552 347 raft_consensus.cc:2454] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:03.866037 424 leader_election.cc:302] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, cfd8971ae77b41e1aa9a26d7f0cd6a9b; no voters:
I20260810 19:32:03.866401 779 raft_consensus.cc:2781] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:03.866536 779 raft_consensus.cc:482] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:03.866612 779 raft_consensus.cc:3037] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.869025 779 raft_consensus.cc:504] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.869846 779 leader_election.cc:288] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Requested vote from peers cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:03.870393 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c68939cdb24224bf84d419b4664bd6" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b"
I20260810 19:32:03.870518 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c68939cdb24224bf84d419b4664bd6" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3"
I20260810 19:32:03.870644 347 raft_consensus.cc:3037] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.870720 656 raft_consensus.cc:3037] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:03.873659 347 raft_consensus.cc:2454] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:03.874111 656 raft_consensus.cc:2454] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:03.874176 424 leader_election.cc:302] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, cfd8971ae77b41e1aa9a26d7f0cd6a9b; no voters:
I20260810 19:32:03.874577 779 raft_consensus.cc:2781] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:03.874742 779 raft_consensus.cc:686] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 1 LEADER]: Becoming Leader. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Running, Role: LEADER
I20260810 19:32:03.875037 779 consensus_queue.cc:237] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:03.878093 32637 catalog_manager.cc:5495] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c931e243575460cb23c7a5db97410b7 (127.29.233.130). New cstate: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:03.953951 347 raft_consensus.cc:1261] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:03.953931 656 raft_consensus.cc:1261] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:03.954602 779 consensus_queue.cc:1035] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:03.955317 779 consensus_queue.cc:1035] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 19:32:03.978945 760 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 19:32:05.575505 32639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:05.575804 32639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:05.580250 32639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Sending DeleteTablet for 3 replicas of tablet 43c68939cdb24224bf84d419b4664bd6
I20260810 19:32:05.581069 481 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c68939cdb24224bf84d419b4664bd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:05 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:45794
I20260810 19:32:05.581179 324 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c68939cdb24224bf84d419b4664bd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:05 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:60180
I20260810 19:32:05.581211 633 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c68939cdb24224bf84d419b4664bd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:05 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52900
I20260810 19:32:05.593595 813 tablet_replica.cc:330] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: stopping tablet replica
I20260810 19:32:05.594060 813 raft_consensus.cc:2229] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:05.594100 815 tablet_replica.cc:330] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: stopping tablet replica
I20260810 19:32:05.594447 815 raft_consensus.cc:2229] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:05.594731 813 raft_consensus.cc:2258] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:05.594833 815 raft_consensus.cc:2258] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:05.596136 815 ts_tablet_manager.cc:1905] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:05.596381 813 ts_tablet_manager.cc:1905] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:05.597939 814 tablet_replica.cc:330] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: stopping tablet replica
I20260810 19:32:05.598148 814 raft_consensus.cc:2229] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:05.598424 814 raft_consensus.cc:2258] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:05.601877 813 ts_tablet_manager.cc:1918] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 19:32:05.602061 813 log.cc:1196] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/43c68939cdb24224bf84d419b4664bd6
I20260810 19:32:05.602627 813 ts_tablet_manager.cc:1939] T 43c68939cdb24224bf84d419b4664bd6 P 0c931e243575460cb23c7a5db97410b7: Deleting consensus metadata
I20260810 19:32:05.603153 814 ts_tablet_manager.cc:1905] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:05.603888 815 ts_tablet_manager.cc:1918] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 19:32:05.603881 32622 catalog_manager.cc:4841] TS 0c931e243575460cb23c7a5db97410b7 (127.29.233.130:44053): tablet 43c68939cdb24224bf84d419b4664bd6 (table test-table [id=36fb0813437b4e55891ea5a5e266771d]) successfully deleted
I20260810 19:32:05.604084 815 log.cc:1196] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/43c68939cdb24224bf84d419b4664bd6
I20260810 19:32:05.604569 815 ts_tablet_manager.cc:1939] T 43c68939cdb24224bf84d419b4664bd6 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting consensus metadata
I20260810 19:32:05.605904 32621 catalog_manager.cc:4841] TS cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225): tablet 43c68939cdb24224bf84d419b4664bd6 (table test-table [id=36fb0813437b4e55891ea5a5e266771d]) successfully deleted
I20260810 19:32:05.608608 814 ts_tablet_manager.cc:1918] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 19:32:05.608777 814 log.cc:1196] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/43c68939cdb24224bf84d419b4664bd6
I20260810 19:32:05.609261 814 ts_tablet_manager.cc:1939] T 43c68939cdb24224bf84d419b4664bd6 P 270318e72d2345c89d43d493ff66eec3: Deleting consensus metadata
I20260810 19:32:05.610378 32624 catalog_manager.cc:4841] TS 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341): tablet 43c68939cdb24224bf84d419b4664bd6 (table test-table [id=36fb0813437b4e55891ea5a5e266771d]) successfully deleted
Aug 10 19:32:05 dist-test-slave-tm1g krb5kdc[32585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 19:32:05 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:05.650096 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:05.635549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.29.233.190:45115)
0810 19:32:05.636169 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:05.636176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:05.636202 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:32:05.636438 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:05.636442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:05.636508 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:05.636628 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:05.637990 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:05.638906 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:05.640188 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:05.640592 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:05.640743 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:05.645139 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:05.645144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:05.645147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:05.645183 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:05.647851 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:05.648486 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:05.648490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:05.648492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:05.648623 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:05.648939 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:05.648942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:05.648944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:05.649200 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:05.649392 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:05.649650 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 19:32:05.649826 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":309,"threads_started":1}
I20260810 19:32:05.653694 32639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:05.654358 32639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:05.661072 324 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac46f06e24246f68ac882a7e12348e1 (DEFAULT_TABLE table=test-table [id=28e92123fc4e4894b3eab66a295b2554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:05.661191 481 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac46f06e24246f68ac882a7e12348e1 (DEFAULT_TABLE table=test-table [id=28e92123fc4e4894b3eab66a295b2554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:05.661396 633 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac46f06e24246f68ac882a7e12348e1 (DEFAULT_TABLE table=test-table [id=28e92123fc4e4894b3eab66a295b2554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:05.661615 324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac46f06e24246f68ac882a7e12348e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:05.661641 481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac46f06e24246f68ac882a7e12348e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:05.661835 633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac46f06e24246f68ac882a7e12348e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:05.668087 825 tablet_bootstrap.cc:492] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Bootstrap starting.
I20260810 19:32:05.669499 825 tablet_bootstrap.cc:654] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:05.675249 826 tablet_bootstrap.cc:492] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: Bootstrap starting.
I20260810 19:32:05.675712 827 tablet_bootstrap.cc:492] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: Bootstrap starting.
I20260810 19:32:05.676970 825 tablet_bootstrap.cc:492] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: No bootstrap required, opened a new log
I20260810 19:32:05.677130 825 ts_tablet_manager.cc:1397] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 19:32:05.677316 827 tablet_bootstrap.cc:654] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:05.677440 826 tablet_bootstrap.cc:654] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:05.678046 825 raft_consensus.cc:348] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.678465 825 raft_consensus.cc:374] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:05.678627 825 raft_consensus.cc:729] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd8971ae77b41e1aa9a26d7f0cd6a9b, State: Initialized, Role: FOLLOWER
I20260810 19:32:05.678937 825 consensus_queue.cc:260] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.679669 827 tablet_bootstrap.cc:492] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: No bootstrap required, opened a new log
I20260810 19:32:05.679839 827 ts_tablet_manager.cc:1397] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260810 19:32:05.679844 826 tablet_bootstrap.cc:492] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: No bootstrap required, opened a new log
I20260810 19:32:05.679863 825 ts_tablet_manager.cc:1428] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:05.679988 826 ts_tablet_manager.cc:1397] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260810 19:32:05.680653 827 raft_consensus.cc:348] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.680922 827 raft_consensus.cc:374] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:05.680871 826 raft_consensus.cc:348] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.681033 827 raft_consensus.cc:729] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270318e72d2345c89d43d493ff66eec3, State: Initialized, Role: FOLLOWER
I20260810 19:32:05.681108 826 raft_consensus.cc:374] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:05.681216 826 raft_consensus.cc:729] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Initialized, Role: FOLLOWER
I20260810 19:32:05.681257 827 consensus_queue.cc:260] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.681478 826 consensus_queue.cc:260] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.682137 827 ts_tablet_manager.cc:1428] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 19:32:05.682943 826 ts_tablet_manager.cc:1428] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 19:32:05.954921 738 raft_consensus.cc:482] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:05.955165 738 raft_consensus.cc:504] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.956175 738 leader_election.cc:288] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:05.956849 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac46f06e24246f68ac882a7e12348e1" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3" is_pre_election: true
I20260810 19:32:05.957175 656 raft_consensus.cc:2454] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:05.957175 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac46f06e24246f68ac882a7e12348e1" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" is_pre_election: true
I20260810 19:32:05.957501 347 raft_consensus.cc:2454] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 0.
I20260810 19:32:05.957700 427 leader_election.cc:302] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, 270318e72d2345c89d43d493ff66eec3; no voters:
I20260810 19:32:05.958062 738 raft_consensus.cc:2781] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:05.958225 738 raft_consensus.cc:482] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:05.958355 738 raft_consensus.cc:3037] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:05.961526 738 raft_consensus.cc:504] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.962067 738 leader_election.cc:288] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Requested vote from peers cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:05.962932 347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac46f06e24246f68ac882a7e12348e1" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b"
I20260810 19:32:05.962978 656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac46f06e24246f68ac882a7e12348e1" candidate_uuid: "0c931e243575460cb23c7a5db97410b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3"
I20260810 19:32:05.963222 656 raft_consensus.cc:3037] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:05.963246 347 raft_consensus.cc:3037] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:05.965360 656 raft_consensus.cc:2454] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:05.965519 347 raft_consensus.cc:2454] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c931e243575460cb23c7a5db97410b7 in term 1.
I20260810 19:32:05.965919 427 leader_election.cc:302] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c931e243575460cb23c7a5db97410b7, 270318e72d2345c89d43d493ff66eec3; no voters:
I20260810 19:32:05.966255 738 raft_consensus.cc:2781] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:05.966462 738 raft_consensus.cc:686] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 1 LEADER]: Becoming Leader. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Running, Role: LEADER
I20260810 19:32:05.966820 738 consensus_queue.cc:237] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:05.969973 32638 catalog_manager.cc:5495] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c931e243575460cb23c7a5db97410b7 (127.29.233.130). New cstate: current_term: 1 leader_uuid: "0c931e243575460cb23c7a5db97410b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:06.063871 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.052148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.29.233.130:44053)
0810 19:32:06.058846 (+ 6698us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:06.058850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:06.058872 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:32:06.058929 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:06.058933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:06.058998 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:06.059136 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:06.059583 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.060484 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.061903 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.062453 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.062599 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.063086 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:06.063199 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:06.063304 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:06.063472 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 19:32:06.063555 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6581,"thread_start_us":6459,"threads_started":1}
I20260810 19:32:06.072501 656 raft_consensus.cc:1261] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:06.072501 347 raft_consensus.cc:1261] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Refusing update from remote peer 0c931e243575460cb23c7a5db97410b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:06.073088 738 consensus_queue.cc:1035] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:06.073458 801 consensus_queue.cc:1035] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:06.130353 32637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:06.130580 32637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:06.133342 32637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Sending DeleteTablet for 3 replicas of tablet 1ac46f06e24246f68ac882a7e12348e1
I20260810 19:32:06.134151 480 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac46f06e24246f68ac882a7e12348e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:06 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:45794
I20260810 19:32:06.134199 324 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac46f06e24246f68ac882a7e12348e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:06 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:60180
I20260810 19:32:06.134282 633 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac46f06e24246f68ac882a7e12348e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:06 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52900
I20260810 19:32:06.135980 32637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:06.136835 32637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:06.146708 842 tablet_replica.cc:330] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: stopping tablet replica
I20260810 19:32:06.147008 842 raft_consensus.cc:2229] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:06.146996 325 tablet_service.cc:1467] Processing CreateTablet for tablet 7a45e12c9daf46888419a70d09291385 (DEFAULT_TABLE table=test-table [id=a3511c26d3694d3db0a0cda42af8ece7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:06.147372 843 tablet_replica.cc:330] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: stopping tablet replica
I20260810 19:32:06.147460 842 raft_consensus.cc:2258] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:06.147509 325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a45e12c9daf46888419a70d09291385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:06.147588 843 raft_consensus.cc:2229] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:06.147743 632 tablet_service.cc:1467] Processing CreateTablet for tablet 7a45e12c9daf46888419a70d09291385 (DEFAULT_TABLE table=test-table [id=a3511c26d3694d3db0a0cda42af8ece7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:06.147930 843 raft_consensus.cc:2258] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:06.148165 632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a45e12c9daf46888419a70d09291385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:06.148576 844 tablet_replica.cc:330] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: stopping tablet replica
I20260810 19:32:06.148720 844 raft_consensus.cc:2229] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:06.148892 844 raft_consensus.cc:2258] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:06.150126 843 ts_tablet_manager.cc:1905] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:06.154222 825 tablet_bootstrap.cc:492] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Bootstrap starting.
I20260810 19:32:06.155160 480 tablet_service.cc:1467] Processing CreateTablet for tablet 7a45e12c9daf46888419a70d09291385 (DEFAULT_TABLE table=test-table [id=a3511c26d3694d3db0a0cda42af8ece7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:06.155715 480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a45e12c9daf46888419a70d09291385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:06.155706 844 ts_tablet_manager.cc:1905] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:06.156044 825 tablet_bootstrap.cc:654] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:06.156724 827 tablet_bootstrap.cc:492] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: Bootstrap starting.
I20260810 19:32:06.158218 825 tablet_bootstrap.cc:492] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: No bootstrap required, opened a new log
I20260810 19:32:06.158192 842 ts_tablet_manager.cc:1905] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:06.158416 825 ts_tablet_manager.cc:1397] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260810 19:32:06.158473 827 tablet_bootstrap.cc:654] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:06.159147 825 raft_consensus.cc:348] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.159410 825 raft_consensus.cc:374] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:06.159515 825 raft_consensus.cc:729] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd8971ae77b41e1aa9a26d7f0cd6a9b, State: Initialized, Role: FOLLOWER
I20260810 19:32:06.159808 825 consensus_queue.cc:260] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.162020 825 ts_tablet_manager.cc:1428] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:06.162962 826 tablet_bootstrap.cc:492] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: Bootstrap starting.
I20260810 19:32:06.164847 826 tablet_bootstrap.cc:654] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:06.166064 842 ts_tablet_manager.cc:1918] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:06.166342 842 log.cc:1196] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/1ac46f06e24246f68ac882a7e12348e1
I20260810 19:32:06.166956 842 ts_tablet_manager.cc:1939] T 1ac46f06e24246f68ac882a7e12348e1 P 0c931e243575460cb23c7a5db97410b7: Deleting consensus metadata
I20260810 19:32:06.168213 32622 catalog_manager.cc:4841] TS 0c931e243575460cb23c7a5db97410b7 (127.29.233.130:44053): tablet 1ac46f06e24246f68ac882a7e12348e1 (table test-table [id=28e92123fc4e4894b3eab66a295b2554]) successfully deleted
I20260810 19:32:06.169011 843 ts_tablet_manager.cc:1918] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:06.169199 827 tablet_bootstrap.cc:492] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: No bootstrap required, opened a new log
I20260810 19:32:06.169400 827 ts_tablet_manager.cc:1397] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260810 19:32:06.169400 844 ts_tablet_manager.cc:1918] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:06.169533 844 log.cc:1196] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/1ac46f06e24246f68ac882a7e12348e1
I20260810 19:32:06.169189 843 log.cc:1196] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/1ac46f06e24246f68ac882a7e12348e1
I20260810 19:32:06.170069 844 ts_tablet_manager.cc:1939] T 1ac46f06e24246f68ac882a7e12348e1 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting consensus metadata
I20260810 19:32:06.170395 843 ts_tablet_manager.cc:1939] T 1ac46f06e24246f68ac882a7e12348e1 P 270318e72d2345c89d43d493ff66eec3: Deleting consensus metadata
I20260810 19:32:06.170415 827 raft_consensus.cc:348] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.170758 827 raft_consensus.cc:374] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:06.170887 827 raft_consensus.cc:729] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270318e72d2345c89d43d493ff66eec3, State: Initialized, Role: FOLLOWER
I20260810 19:32:06.171041 826 tablet_bootstrap.cc:492] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: No bootstrap required, opened a new log
I20260810 19:32:06.171247 826 ts_tablet_manager.cc:1397] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 19:32:06.171221 827 consensus_queue.cc:260] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.171387 32621 catalog_manager.cc:4841] TS cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225): tablet 1ac46f06e24246f68ac882a7e12348e1 (table test-table [id=28e92123fc4e4894b3eab66a295b2554]) successfully deleted
I20260810 19:32:06.171607 32624 catalog_manager.cc:4841] TS 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341): tablet 1ac46f06e24246f68ac882a7e12348e1 (table test-table [id=28e92123fc4e4894b3eab66a295b2554]) successfully deleted
I20260810 19:32:06.171929 827 ts_tablet_manager.cc:1428] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 19:32:06.172711 826 raft_consensus.cc:348] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.172964 826 raft_consensus.cc:374] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:06.173056 826 raft_consensus.cc:729] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c931e243575460cb23c7a5db97410b7, State: Initialized, Role: FOLLOWER
I20260810 19:32:06.173291 826 consensus_queue.cc:260] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.173763 826 ts_tablet_manager.cc:1428] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 19:32:06.444476 832 raft_consensus.cc:482] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:06.444756 832 raft_consensus.cc:504] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.446399 832 leader_election.cc:288] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c931e243575460cb23c7a5db97410b7 (127.29.233.130:44053), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:06.455848 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.446843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53721 (local address 127.29.233.130:44053)
0810 19:32:06.447064 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:06.447073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:06.447970 (+ 897us) server_negotiation.cc:407] Connection header received
0810 19:32:06.448205 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:06.448211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:06.448297 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:06.448412 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:06.449296 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.450522 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.453512 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.454889 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.455060 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.455139 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:06.455452 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 19:32:06.455585 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:32:06.456069 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.446740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:44053 (local address 127.29.233.129:53721)
0810 19:32:06.447837 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:06.447857 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:06.448035 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:06.448459 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:06.448472 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:06.448524 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:06.449124 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.449136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.450678 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.450684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.453334 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.453350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.455103 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.455111 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.455277 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.455306 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 19:32:06.455421 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 19:32:06.455571 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":296,"spinlock_wait_cycles":8320,"thread_start_us":936,"threads_started":1}
I20260810 19:32:06.456106 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.446915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:33293 (local address 127.29.233.131:39341)
0810 19:32:06.447515 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:06.447522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:06.448094 (+ 572us) server_negotiation.cc:407] Connection header received
0810 19:32:06.448234 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:06.448239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:06.448325 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:06.448422 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:06.449294 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.450533 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.453512 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.454886 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.455083 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.455150 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:06.455596 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 19:32:06.455728 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":303,"threads_started":1}
I20260810 19:32:06.456187 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.446775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:39341 (local address 127.29.233.129:33293)
0810 19:32:06.447982 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:06.448005 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:06.448118 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:06.448459 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:06.448465 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:06.448522 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:06.449054 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.449073 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.450678 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.450684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.453334 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.453350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.455299 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.455304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.455433 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.455457 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:32:06.455567 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 19:32:06.455705 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1088,"mutex_wait_us":153,"spinlock_wait_cycles":20736,"thread_start_us":713,"threads_started":1}
I20260810 19:32:06.457103 654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a45e12c9daf46888419a70d09291385" candidate_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3" is_pre_election: true
I20260810 19:32:06.457074 504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a45e12c9daf46888419a70d09291385" candidate_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c931e243575460cb23c7a5db97410b7" is_pre_election: true
I20260810 19:32:06.457379 654 raft_consensus.cc:2454] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfd8971ae77b41e1aa9a26d7f0cd6a9b in term 0.
I20260810 19:32:06.457504 504 raft_consensus.cc:2454] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfd8971ae77b41e1aa9a26d7f0cd6a9b in term 0.
I20260810 19:32:06.457930 32741 leader_election.cc:302] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270318e72d2345c89d43d493ff66eec3, cfd8971ae77b41e1aa9a26d7f0cd6a9b; no voters:
I20260810 19:32:06.458443 832 raft_consensus.cc:2781] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:06.458570 832 raft_consensus.cc:482] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:06.458683 832 raft_consensus.cc:3037] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:06.461025 832 raft_consensus.cc:504] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.461673 832 leader_election.cc:288] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [CANDIDATE]: Term 1 election: Requested vote from peers 0c931e243575460cb23c7a5db97410b7 (127.29.233.130:44053), 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341)
I20260810 19:32:06.462157 504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a45e12c9daf46888419a70d09291385" candidate_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c931e243575460cb23c7a5db97410b7"
I20260810 19:32:06.462195 654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a45e12c9daf46888419a70d09291385" candidate_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270318e72d2345c89d43d493ff66eec3"
I20260810 19:32:06.462393 504 raft_consensus.cc:3037] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:06.462411 654 raft_consensus.cc:3037] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:06.464450 654 raft_consensus.cc:2454] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfd8971ae77b41e1aa9a26d7f0cd6a9b in term 1.
I20260810 19:32:06.464598 504 raft_consensus.cc:2454] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfd8971ae77b41e1aa9a26d7f0cd6a9b in term 1.
I20260810 19:32:06.464919 32741 leader_election.cc:302] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270318e72d2345c89d43d493ff66eec3, cfd8971ae77b41e1aa9a26d7f0cd6a9b; no voters:
I20260810 19:32:06.465204 832 raft_consensus.cc:2781] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:06.465665 832 raft_consensus.cc:686] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 LEADER]: Becoming Leader. State: Replica: cfd8971ae77b41e1aa9a26d7f0cd6a9b, State: Running, Role: LEADER
I20260810 19:32:06.466055 832 consensus_queue.cc:237] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } }
I20260810 19:32:06.469373 32637 catalog_manager.cc:5495] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b reported cstate change: term changed from 0 to 1, leader changed from <none> to cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129). New cstate: current_term: 1 leader_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd8971ae77b41e1aa9a26d7f0cd6a9b" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 34225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:06.546747 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.539757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60182 (local address 127.29.233.129:34225)
0810 19:32:06.540344 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:06.540350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:06.540373 (+ 23us) server_negotiation.cc:407] Connection header received
0810 19:32:06.540493 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:06.540498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:06.540589 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:06.540755 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:06.541132 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.542432 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.543980 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.544645 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.544774 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.545229 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:06.546031 (+ 802us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:06.546167 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:06.546258 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 19:32:06.546361 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":304,"threads_started":1}
I20260810 19:32:06.563611 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.557628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:42111 (local address 127.29.233.130:44053)
0810 19:32:06.557827 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:06.557834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:06.558556 (+ 722us) server_negotiation.cc:407] Connection header received
0810 19:32:06.558724 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:06.558729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:06.558830 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:06.558928 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:06.559516 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.560449 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.561974 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.562451 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.562553 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.563000 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:06.563109 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:06.563175 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:06.563300 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 19:32:06.563368 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:32:06.563654 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.557603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:44053 (local address 127.29.233.129:42111)
0810 19:32:06.558459 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:06.558477 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:06.558595 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:06.558954 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:06.558958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:06.558968 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:06.559340 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.559349 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.560584 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.560588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.561834 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.561848 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.562639 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.562644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.562775 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.562836 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:06.563195 (+ 359us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:06.563205 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 19:32:06.563264 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 19:32:06.563368 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":430,"threads_started":1}
I20260810 19:32:06.571678 654 raft_consensus.cc:1261] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Refusing update from remote peer cfd8971ae77b41e1aa9a26d7f0cd6a9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:06.572131 504 raft_consensus.cc:1261] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Refusing update from remote peer cfd8971ae77b41e1aa9a26d7f0cd6a9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:06.572320 851 consensus_queue.cc:1035] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "270318e72d2345c89d43d493ff66eec3" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:06.572870 832 consensus_queue.cc:1035] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c931e243575460cb23c7a5db97410b7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:06.600251 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.594059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:48353 (local address 127.29.233.129:34225)
0810 19:32:06.594254 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:06.594260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:06.594938 (+ 678us) server_negotiation.cc:407] Connection header received
0810 19:32:06.595083 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:06.595090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:06.595172 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:06.595247 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:06.595828 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.596720 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.598621 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.599205 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.599310 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.599715 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:06.599814 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:06.599882 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:06.599987 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 19:32:06.600058 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 19:32:06.600415 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:06.594138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:34225 (local address 127.29.233.130:48353)
0810 19:32:06.594856 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:06.594872 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:06.594980 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:06.595277 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:06.595280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:06.595359 (+ 79us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:06.595697 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.595707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.596864 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.596868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.598485 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:06.598498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:06.599404 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:06.599408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:06.599538 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:06.599574 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:06.599899 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:06.599907 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 19:32:06.599961 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 19:32:06.600061 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":359,"threads_started":1}
W20260810 19:32:06.601630 797 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 19:32:08.206506 32637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:08.206782 32637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:08.211444 32637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04cca5fe45e94708a84aff8a35f3d8f8: Sending DeleteTablet for 3 replicas of tablet 7a45e12c9daf46888419a70d09291385
I20260810 19:32:08.212416 325 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a45e12c9daf46888419a70d09291385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:08 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:60180
I20260810 19:32:08.212430 480 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a45e12c9daf46888419a70d09291385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:08 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:45794
I20260810 19:32:08.212529 632 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a45e12c9daf46888419a70d09291385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:08 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52900
I20260810 19:32:08.213194 892 tablet_replica.cc:330] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: stopping tablet replica
I20260810 19:32:08.213445 892 raft_consensus.cc:2229] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:08.213573 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32721
I20260810 19:32:08.213732 892 raft_consensus.cc:2258] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:08.214367 889 tablet_replica.cc:330] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: stopping tablet replica
I20260810 19:32:08.214751 889 raft_consensus.cc:2229] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:08.214972 892 ts_tablet_manager.cc:1905] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:08.215533 889 raft_consensus.cc:2258] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:08.217331 889 ts_tablet_manager.cc:1905] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:08.223596 892 ts_tablet_manager.cc:1918] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 19:32:08.223794 892 log.cc:1196] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/7a45e12c9daf46888419a70d09291385
I20260810 19:32:08.224251 892 ts_tablet_manager.cc:1939] T 7a45e12c9daf46888419a70d09291385 P 0c931e243575460cb23c7a5db97410b7: Deleting consensus metadata
I20260810 19:32:08.225631 32622 catalog_manager.cc:4841] TS 0c931e243575460cb23c7a5db97410b7 (127.29.233.130:44053): tablet 7a45e12c9daf46888419a70d09291385 (table test-table [id=a3511c26d3694d3db0a0cda42af8ece7]) successfully deleted
I20260810 19:32:08.226809 893 tablet_replica.cc:330] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: stopping tablet replica
I20260810 19:32:08.227099 893 raft_consensus.cc:2229] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:08.227439 893 raft_consensus.cc:2258] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:08.227972 889 ts_tablet_manager.cc:1918] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 19:32:08.228255 889 log.cc:1196] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/7a45e12c9daf46888419a70d09291385
I20260810 19:32:08.228847 893 ts_tablet_manager.cc:1905] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:08.228971 889 ts_tablet_manager.cc:1939] T 7a45e12c9daf46888419a70d09291385 P cfd8971ae77b41e1aa9a26d7f0cd6a9b: Deleting consensus metadata
I20260810 19:32:08.230612 32621 catalog_manager.cc:4841] TS cfd8971ae77b41e1aa9a26d7f0cd6a9b (127.29.233.129:34225): tablet 7a45e12c9daf46888419a70d09291385 (table test-table [id=a3511c26d3694d3db0a0cda42af8ece7]) successfully deleted
I20260810 19:32:08.235329 893 ts_tablet_manager.cc:1918] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 19:32:08.235488 893 log.cc:1196] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/7a45e12c9daf46888419a70d09291385
I20260810 19:32:08.235986 893 ts_tablet_manager.cc:1939] T 7a45e12c9daf46888419a70d09291385 P 270318e72d2345c89d43d493ff66eec3: Deleting consensus metadata
I20260810 19:32:08.237104 32624 catalog_manager.cc:4841] TS 270318e72d2345c89d43d493ff66eec3 (127.29.233.131:39341): tablet 7a45e12c9daf46888419a70d09291385 (table test-table [id=a3511c26d3694d3db0a0cda42af8ece7]) successfully deleted
Aug 10 19:32:08 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:08.253197 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:08.216505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.29.233.129:34225)
0810 19:32:08.230861 (+ 14356us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:08.230866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:08.230886 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:08.230948 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:08.230951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:08.231016 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:08.231128 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:08.233249 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:08.234024 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:08.236161 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:08.236591 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:08.236700 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:08.247061 (+ 10361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:08.247079 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:08.247097 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:08.247146 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:08.250209 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:08.251157 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:08.251164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:08.251167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:08.251330 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:08.251682 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:08.251685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:08.251687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:08.252074 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:08.252290 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:08.252721 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 19:32:08.252905 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14195,"thread_start_us":14064,"threads_started":1}
W20260810 19:32:08.253916 396 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60196
W20260810 19:32:08.254604 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32721: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:32:08.254812 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 32721
I20260810 19:32:08.282904 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 405
Aug 10 19:32:08 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:32:08.300056 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:08.283897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.29.233.130:44053)
0810 19:32:08.284476 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:08.284480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:08.284496 (+ 16us) server_negotiation.cc:407] Connection header received
0810 19:32:08.284551 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:08.284554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:08.284620 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:08.284718 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:08.286269 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:08.287194 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:08.288572 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:08.288994 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:08.289104 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:08.293668 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:08.293684 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:08.293702 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:08.293745 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:08.297065 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:08.297879 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:08.297887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:08.297890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:08.298057 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:08.298576 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:08.298580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:08.298582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:08.298974 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:08.299244 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:08.299578 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 19:32:08.299772 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":313,"threads_started":1}
W20260810 19:32:08.300781 552 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45832
W20260810 19:32:08.301538 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 405: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:32:08.301725 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 405
I20260810 19:32:08.333520 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 562
Aug 10 19:32:08 dist-test-slave-tm1g krb5kdc[32585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:32:08.351528 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:08.334487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.29.233.131:39341)
0810 19:32:08.335077 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:08.335081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:08.335098 (+ 17us) server_negotiation.cc:407] Connection header received
0810 19:32:08.335155 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:08.335159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:08.335221 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:08.335336 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:08.336799 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:08.337841 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:08.339448 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:08.339901 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:08.340029 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:08.345122 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:08.345140 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:08.345157 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:08.345196 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:08.348390 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:08.349303 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:08.349311 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:08.349314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:08.349477 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:08.349951 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:08.349955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:08.349957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:08.350490 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:08.350737 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:08.351060 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 19:32:08.351240 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":345,"threads_started":1}
W20260810 19:32:08.352241 704 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52904
W20260810 19:32:08.352970 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 562: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:32:08.353197 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 562
I20260810 19:32:08.375313 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 32603
I20260810 19:32:08.376664 32684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:08.497784 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 32603
2026-08-10T19:32:08Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9465 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:08 dist-test-slave-tm1g krb5kdc[902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:08 dist-test-slave-tm1g krb5kdc[902](info): set up 2 sockets
Aug 10 19:32:08 dist-test-slave-tm1g krb5kdc[902](info): commencing operation
krb5kdc: starting...
W20260810 19:32:09.670636 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:09 dist-test-slave-tm1g krb5kdc[902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:09.910578 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42627
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:42627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:10.248199 916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:10.248556 916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:10.248718 916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:10.255986 916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:10.256134 916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:10.256237 916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:10.256350 916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:10.256428 916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:10.266131 916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:42627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:10.267495 916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:10.268803 916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:10.280604 925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:10.280651 927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:10.280697 923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:10.281611 916 server_base.cc:1034] running on GCE node
I20260810 19:32:10.282493 916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:10.283926 916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:10.285207 916 hybrid_clock.cc:648] HybridClock initialized: now 1786390330285168 us; error 40 us; skew 500 ppm
I20260810 19:32:10.285550 916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:10 dist-test-slave-tm1g krb5kdc[902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390330, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:10.288723 916 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:10.290266 916 webserver.cc:460] Webserver started at http://127.29.233.190:41425/ using document root <none> and password file <none>
I20260810 19:32:10.291052 916 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:10.291182 916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:10.291525 916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:10.294786 916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538"
format_stamp: "Formatted at 2026-08-10 19:32:10 on dist-test-slave-tm1g"
server_key: "bea2e3995f2d5ddda101028c204a1aed"
server_key_iv: "405fb9393249cad6f516f80effbb9f53"
server_key_version: "encryptionkey@0"
I20260810 19:32:10.295575 916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538"
format_stamp: "Formatted at 2026-08-10 19:32:10 on dist-test-slave-tm1g"
server_key: "bea2e3995f2d5ddda101028c204a1aed"
server_key_iv: "405fb9393249cad6f516f80effbb9f53"
server_key_version: "encryptionkey@0"
I20260810 19:32:10.300881 916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 19:32:10.304382 934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:10.305744 916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:10.305932 916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538"
format_stamp: "Formatted at 2026-08-10 19:32:10 on dist-test-slave-tm1g"
server_key: "bea2e3995f2d5ddda101028c204a1aed"
server_key_iv: "405fb9393249cad6f516f80effbb9f53"
server_key_version: "encryptionkey@0"
I20260810 19:32:10.306145 916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:10.328631 916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:10.333271 916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:10.333602 916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:10.352978 916 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:42627
I20260810 19:32:10.353009 994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:42627 every 8 connection(s)
I20260810 19:32:10.354444 916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:10.358275 995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:10.364672 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 916
I20260810 19:32:10.364948 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:10.365852 30630 external_mini_cluster.cc:1428] Setting key 9488c9b3750777f78b2b28a60a6030c7
I20260810 19:32:10.369861 995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538: Bootstrap starting.
I20260810 19:32:10.373724 995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:10.374876 995 log.cc:824] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:10.377946 995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538: No bootstrap required, opened a new log
Aug 10 19:32:10 dist-test-slave-tm1g krb5kdc[902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:10.386987 995 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42627 } }
I20260810 19:32:10.387401 995 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:10.387540 995 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbbc66f4cb74ef1b87c46d3ffb84538, State: Initialized, Role: FOLLOWER
I20260810 19:32:10.388339 995 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42627 } }
I20260810 19:32:10.388747 995 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:10.388907 995 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:10.389115 995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:10.389905 998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:10.368092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.29.233.190:42627)
0810 19:32:10.368874 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:10.368886 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:10.368926 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:32:10.370714 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:10.370736 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:10.371092 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:10.371416 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:10.373303 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:10.374728 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:10.376294 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:10.376972 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:10.377109 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:10.382030 (+ 4921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:10.382038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:10.382053 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:10.382101 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:10.385434 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:10.386139 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:10.386151 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:10.386157 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:10.386395 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:10.386783 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:10.386788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:10.386791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:10.387467 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:10.387804 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:10.388227 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 19:32:10.388642 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":364,"threads_started":1}
I20260810 19:32:10.391738 995 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42627 } }
I20260810 19:32:10.392261 995 leader_election.cc:302] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcbbc66f4cb74ef1b87c46d3ffb84538; no voters:
I20260810 19:32:10.392953 995 leader_election.cc:288] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:10.393043 1000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:10.393829 1000 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [term 1 LEADER]: Becoming Leader. State: Replica: bcbbc66f4cb74ef1b87c46d3ffb84538, State: Running, Role: LEADER
I20260810 19:32:10.394467 1000 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42627 } }
I20260810 19:32:10.395172 995 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:10.397200 1001 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42627 } } }
I20260810 19:32:10.397550 1001 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:10.398530 1002 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcbbc66f4cb74ef1b87c46d3ffb84538. Latest consensus state: current_term: 1 leader_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcbbc66f4cb74ef1b87c46d3ffb84538" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42627 } } }
I20260810 19:32:10.398958 1002 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:10.399837 1007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:10.405174 1007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:10.414984 1007 catalog_manager.cc:1347] Generated new cluster ID: 1bb4f0bcc5724a7fada9984cea10389e
I20260810 19:32:10.415105 1007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:10.427246 1007 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:10.429263 1007 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:10.437747 1007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcbbc66f4cb74ef1b87c46d3ffb84538: Generated new TSK 0
I20260810 19:32:10.438815 1007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:32:10.501971 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42627
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:10.838683 1022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:10.839008 1022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:10.839186 1022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:10.846514 1022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:10.846664 1022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:10.846858 1022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:10.856839 1022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:10.858256 1022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:10.859654 1022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:10.873289 1027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:10.873276 1029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:10.873358 1031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:10.873950 1022 server_base.cc:1034] running on GCE node
I20260810 19:32:10.874657 1022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:10.875725 1022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:10.876976 1022 hybrid_clock.cc:648] HybridClock initialized: now 1786390330876931 us; error 64 us; skew 500 ppm
I20260810 19:32:10.877337 1022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:10 dist-test-slave-tm1g krb5kdc[902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390330, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:10.880591 1022 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:32:10.882025 1022 webserver.cc:460] Webserver started at http://127.29.233.129:41819/ using document root <none> and password file <none>
I20260810 19:32:10.882798 1022 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:10.882956 1022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:10.883283 1022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:10.886965 1022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "d82a182a2adc4ce19c646f257523c136"
format_stamp: "Formatted at 2026-08-10 19:32:10 on dist-test-slave-tm1g"
server_key: "bf92349f84a32c97742393ba8e34b53d"
server_key_iv: "782f9040f3a6b1de64f0c9281055c53d"
server_key_version: "encryptionkey@0"
I20260810 19:32:10.887802 1022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "d82a182a2adc4ce19c646f257523c136"
format_stamp: "Formatted at 2026-08-10 19:32:10 on dist-test-slave-tm1g"
server_key: "bf92349f84a32c97742393ba8e34b53d"
server_key_iv: "782f9040f3a6b1de64f0c9281055c53d"
server_key_version: "encryptionkey@0"
I20260810 19:32:10.893319 1022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:32:10.897347 1038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:10.898828 1022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:10.899011 1022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "d82a182a2adc4ce19c646f257523c136"
format_stamp: "Formatted at 2026-08-10 19:32:10 on dist-test-slave-tm1g"
server_key: "bf92349f84a32c97742393ba8e34b53d"
server_key_iv: "782f9040f3a6b1de64f0c9281055c53d"
server_key_version: "encryptionkey@0"
I20260810 19:32:10.899240 1022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:10.919731 1022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:10.924044 1022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:10.924366 1022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:10.925402 1022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:10.926868 1022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:10.926975 1022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:10.927132 1022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:10.927207 1022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:10.962918 1022 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:45867
I20260810 19:32:10.962960 1169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:45867 every 8 connection(s)
I20260810 19:32:10.964290 1022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:10.967610 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 1022
I20260810 19:32:10.967885 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:10.968838 30630 external_mini_cluster.cc:1428] Setting key 95b81eb5ae8906bd5e09b990a41e9f17
Aug 10 19:32:10 dist-test-slave-tm1g krb5kdc[902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390330, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:10.989809 1172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:10.967240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:39641 (local address 127.29.233.190:42627)
0810 19:32:10.968414 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:10.968425 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:10.969165 (+ 740us) server_negotiation.cc:407] Connection header received
0810 19:32:10.971508 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:10.971513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:10.971594 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:10.971692 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:10.973907 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:10.974762 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:10.977361 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:10.977773 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:10.977891 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:10.982109 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:10.982118 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:10.982126 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:10.982176 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:10.985942 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:10.986860 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:10.986869 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:10.986875 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:10.987037 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:10.987587 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:10.987595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:10.987601 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:10.988005 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:10.988181 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:10.988979 (+ 798us) server_negotiation.cc:299] Negotiation successful
0810 19:32:10.989283 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":991,"thread_start_us":844,"threads_started":1}
I20260810 19:32:10.990680 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:10.967934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42627 (local address 127.29.233.129:39641)
0810 19:32:10.968937 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:10.969024 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:10.971194 (+ 2170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:10.972003 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:10.972018 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:10.972816 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:10.973681 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:10.973711 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:10.974924 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:10.974932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:10.977148 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:10.977168 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:10.978033 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:10.978043 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:10.978402 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:10.979152 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:10.979192 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:10.981855 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:10.986116 (+ 4261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:10.986124 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:10.986140 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:10.986717 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:10.987199 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:10.987203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:10.987206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:10.987442 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:10.988157 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:10.988163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:10.988545 (+ 382us) client_negotiation.cc:769] Sending connection context
0810 19:32:10.988804 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 19:32:10.989289 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":417,"threads_started":1}
I20260810 19:32:10.992478 1170 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42627
I20260810 19:32:10.992813 1170 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:10.993615 1170 heartbeater.cc:511] Master 127.29.233.190:42627 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
I20260810 19:32:10.996232 957 ts_manager.cc:194] Registered new tserver with Master: d82a182a2adc4ce19c646f257523c136 (127.29.233.129:45867)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
I20260810 19:32:10.998062 957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:39641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:32:11.030167 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42627
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:11.384083 1176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:11.384421 1176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:11.384598 1176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:11.392091 1176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:11.392241 1176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:11.392426 1176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:11.403643 1176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:11.405140 1176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:11.406594 1176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:11.420499 1185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:11.420598 1183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:11.420858 1182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:11.421573 1176 server_base.cc:1034] running on GCE node
I20260810 19:32:11.422212 1176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:11.423200 1176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:11.424441 1176 hybrid_clock.cc:648] HybridClock initialized: now 1786390331424387 us; error 48 us; skew 500 ppm
I20260810 19:32:11.424777 1176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:11 dist-test-slave-tm1g krb5kdc[902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:11.428185 1176 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:32:11.429802 1176 webserver.cc:460] Webserver started at http://127.29.233.130:35995/ using document root <none> and password file <none>
I20260810 19:32:11.430630 1176 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:11.430781 1176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:11.431123 1176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:11.434103 1176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "b34885782e5e41e688d83865ea62941e"
format_stamp: "Formatted at 2026-08-10 19:32:11 on dist-test-slave-tm1g"
server_key: "ddf0513b8a5b552bb4868e0dde5f8b65"
server_key_iv: "dc9fa55b38c9e045589a86d55e9e42ea"
server_key_version: "encryptionkey@0"
I20260810 19:32:11.434969 1176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "b34885782e5e41e688d83865ea62941e"
format_stamp: "Formatted at 2026-08-10 19:32:11 on dist-test-slave-tm1g"
server_key: "ddf0513b8a5b552bb4868e0dde5f8b65"
server_key_iv: "dc9fa55b38c9e045589a86d55e9e42ea"
server_key_version: "encryptionkey@0"
I20260810 19:32:11.440264 1176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:32:11.443879 1193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:11.445708 1176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:32:11.445946 1176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "b34885782e5e41e688d83865ea62941e"
format_stamp: "Formatted at 2026-08-10 19:32:11 on dist-test-slave-tm1g"
server_key: "ddf0513b8a5b552bb4868e0dde5f8b65"
server_key_iv: "dc9fa55b38c9e045589a86d55e9e42ea"
server_key_version: "encryptionkey@0"
I20260810 19:32:11.446190 1176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:11.467746 1176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:11.472005 1176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:11.472329 1176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:11.473263 1176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:11.474607 1176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:11.474684 1176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:11.474768 1176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:11.474824 1176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:11.508765 1176 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:44295
I20260810 19:32:11.508812 1322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:44295 every 8 connection(s)
I20260810 19:32:11.510147 1176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:11.513131 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 1176
I20260810 19:32:11.513353 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:11.513875 30630 external_mini_cluster.cc:1428] Setting key f7da7b11a0717f019eaca427f475a14f
Aug 10 19:32:11 dist-test-slave-tm1g krb5kdc[902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:11.532902 1325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:11.512940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:38061 (local address 127.29.233.190:42627)
0810 19:32:11.513491 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:11.513498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:11.514784 (+ 1286us) server_negotiation.cc:407] Connection header received
0810 19:32:11.517036 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:11.517043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:11.517126 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:11.517248 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:11.518879 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:11.520052 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:11.521783 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:11.522585 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:11.522733 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:11.526510 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:11.526516 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:11.526519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:11.526554 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:11.529289 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:11.530294 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:11.530338 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:11.530348 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:11.530511 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:11.531036 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:11.531040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:11.531042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:11.531319 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:11.531462 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:11.532428 (+ 966us) server_negotiation.cc:299] Negotiation successful
0810 19:32:11.532622 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":244,"threads_started":1}
I20260810 19:32:11.534605 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:11.513565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42627 (local address 127.29.233.130:38061)
0810 19:32:11.514576 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:11.514623 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:11.516718 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:11.517458 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:11.517466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:11.518020 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:11.518696 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:11.518714 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:11.520294 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:11.520298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:11.521629 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:11.521642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:11.522809 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:11.522814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:11.523024 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:11.523788 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:11.523818 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:11.526203 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:11.529440 (+ 3237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:11.529452 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:11.529475 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:11.530164 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:11.530637 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:11.530645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:11.530651 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:11.530909 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:11.531451 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:11.531462 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:11.531950 (+ 488us) client_negotiation.cc:769] Sending connection context
0810 19:32:11.532301 (+ 351us) client_negotiation.cc:240] Negotiation successful
0810 19:32:11.532950 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":356,"threads_started":1}
I20260810 19:32:11.536170 1323 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42627
I20260810 19:32:11.536489 1323 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:11.537220 1323 heartbeater.cc:511] Master 127.29.233.190:42627 requested a full tablet report, sending...
I20260810 19:32:11.538770 957 ts_manager.cc:194] Registered new tserver with Master: b34885782e5e41e688d83865ea62941e (127.29.233.130:44295)
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
I20260810 19:32:11.539652 957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:38061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:32:11.570493 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42627
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:11.911533 1329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:11.911875 1329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:11.912048 1329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:11.919509 1329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:11.919679 1329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:11.919857 1329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:11.929919 1329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:11.931437 1329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:11.932834 1329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:11.946666 1336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:11.946671 1335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:11.946862 1338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:11.947530 1329 server_base.cc:1034] running on GCE node
I20260810 19:32:11.948153 1329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:11.949138 1329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:11.950402 1329 hybrid_clock.cc:648] HybridClock initialized: now 1786390331950328 us; error 66 us; skew 500 ppm
I20260810 19:32:11.950819 1329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:11 dist-test-slave-tm1g krb5kdc[902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:11.954152 1329 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:32:11.955641 1329 webserver.cc:460] Webserver started at http://127.29.233.131:35363/ using document root <none> and password file <none>
I20260810 19:32:11.956375 1329 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:11.956554 1329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:11.956884 1329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:11.959913 1329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "f9e3195506d24c328774d61c83236713"
format_stamp: "Formatted at 2026-08-10 19:32:11 on dist-test-slave-tm1g"
server_key: "d1cef33daa96259d863b715ce91e0869"
server_key_iv: "10849b8adfef734dfe04c7b8d190d4a4"
server_key_version: "encryptionkey@0"
I20260810 19:32:11.960738 1329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "f9e3195506d24c328774d61c83236713"
format_stamp: "Formatted at 2026-08-10 19:32:11 on dist-test-slave-tm1g"
server_key: "d1cef33daa96259d863b715ce91e0869"
server_key_iv: "10849b8adfef734dfe04c7b8d190d4a4"
server_key_version: "encryptionkey@0"
I20260810 19:32:11.966292 1329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260810 19:32:11.969861 1346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:11.971302 1329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:11.971536 1329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "f9e3195506d24c328774d61c83236713"
format_stamp: "Formatted at 2026-08-10 19:32:11 on dist-test-slave-tm1g"
server_key: "d1cef33daa96259d863b715ce91e0869"
server_key_iv: "10849b8adfef734dfe04c7b8d190d4a4"
server_key_version: "encryptionkey@0"
I20260810 19:32:11.971755 1329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:11.986212 1329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:11.990370 1329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:11.990674 1329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:11.991667 1329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:11.993028 1329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:11.993136 1329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:11.993294 1329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:11.993369 1329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:12.001767 1170 heartbeater.cc:503] Master 127.29.233.190:42627 was elected leader, sending a full tablet report...
I20260810 19:32:12.028685 1329 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:37009
I20260810 19:32:12.028735 1472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:37009 every 8 connection(s)
I20260810 19:32:12.030017 1329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:12.032960 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 1329
I20260810 19:32:12.033175 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:12.033656 30630 external_mini_cluster.cc:1428] Setting key fbe4d91780bc0fb7ac115b76c3342243
Aug 10 19:32:12 dist-test-slave-tm1g krb5kdc[902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:12.051357 1325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:12.033068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:48471 (local address 127.29.233.190:42627)
0810 19:32:12.033301 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:12.033308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:12.034701 (+ 1393us) server_negotiation.cc:407] Connection header received
0810 19:32:12.036756 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:12.036763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:12.036856 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:12.036946 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:12.038373 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:12.039135 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:12.040740 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:12.041144 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:12.041240 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:12.044906 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:12.044911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:12.044915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:12.044949 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:12.047690 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:12.048478 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:12.048485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:12.048488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:12.048675 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:12.049264 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:12.049269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:12.049273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:12.049580 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:12.049759 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:12.050630 (+ 871us) server_negotiation.cc:299] Negotiation successful
0810 19:32:12.050869 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 19:32:12.052291 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:12.033621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42627 (local address 127.29.233.131:48471)
0810 19:32:12.034536 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:12.034589 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:12.036497 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:12.037120 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:12.037128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:12.037628 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:12.038178 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:12.038194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:12.039346 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:12.039351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:12.040609 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:12.040621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:12.041408 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:12.041415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:12.041644 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:12.042396 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:12.042432 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:12.044692 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:12.047850 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:12.047858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:12.047882 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:12.048357 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:12.048811 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:12.048818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:12.048821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:12.049134 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:12.049723 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:12.049731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:12.050195 (+ 464us) client_negotiation.cc:769] Sending connection context
0810 19:32:12.050549 (+ 354us) client_negotiation.cc:240] Negotiation successful
0810 19:32:12.051151 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":368,"threads_started":1}
I20260810 19:32:12.053586 1473 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42627
I20260810 19:32:12.053903 1473 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:12.054702 1473 heartbeater.cc:511] Master 127.29.233.190:42627 requested a full tablet report, sending...
I20260810 19:32:12.056098 957 ts_manager.cc:194] Registered new tserver with Master: f9e3195506d24c328774d61c83236713 (127.29.233.131:37009)
I20260810 19:32:12.056929 957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:48471
I20260810 19:32:12.061558 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 19:32:12.104326 1325 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:12.101454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55432 (local address 127.29.233.190:42627)
0810 19:32:12.101687 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:12.101696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:12.102098 (+ 402us) server_negotiation.cc:407] Connection header received
0810 19:32:12.102386 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:12.102390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:12.102469 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:12.102584 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:12.103972 (+ 1388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55432: BlockingRecv error: recv got EOF from 127.0.0.1:55432 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:32:12.107625 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1022
W20260810 19:32:12.110464 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1022: Not authorized: Client connection negotiation failed: client connection to 127.29.233.129:45867: server requires authentication, but client does not have Kerberos credentials available
I20260810 19:32:12.110652 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 1022
I20260810 19:32:12.130879 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1176
W20260810 19:32:12.133677 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1176: Not authorized: Client connection negotiation failed: client connection to 127.29.233.130:44295: server requires authentication, but client does not have Kerberos credentials available
I20260810 19:32:12.133883 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 1176
I20260810 19:32:12.152647 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1329
W20260810 19:32:12.155798 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1329: Not authorized: Client connection negotiation failed: client connection to 127.29.233.131:37009: server requires authentication, but client does not have Kerberos credentials available
I20260810 19:32:12.155979 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 1329
I20260810 19:32:12.174806 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 916
I20260810 19:32:12.176117 989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:12.281697 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 916
2026-08-10T19:32:12Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3779 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:12 dist-test-slave-tm1g krb5kdc[1491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:12 dist-test-slave-tm1g krb5kdc[1491](info): set up 2 sockets
Aug 10 19:32:12 dist-test-slave-tm1g krb5kdc[1491](info): commencing operation
krb5kdc: starting...
W20260810 19:32:13.438126 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:13 dist-test-slave-tm1g krb5kdc[1491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:13.675161 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:34997
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:34997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:14.018080 1506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:14.018428 1506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:14.018522 1506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:14.025799 1506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:14.025918 1506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:14.025966 1506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:14.026017 1506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:14.026070 1506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:14.036231 1506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:34997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:34997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:14.037573 1506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:14.038849 1506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:14.050849 1512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:14.050879 1515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:14.050974 1513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:14.052623 1506 server_base.cc:1034] running on GCE node
I20260810 19:32:14.053510 1506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:14.055076 1506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:14.056320 1506 hybrid_clock.cc:648] HybridClock initialized: now 1786390334056281 us; error 35 us; skew 500 ppm
I20260810 19:32:14.056736 1506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:14 dist-test-slave-tm1g krb5kdc[1491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390334, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:14.060178 1506 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:14.061820 1506 webserver.cc:460] Webserver started at http://127.29.233.190:37393/ using document root <none> and password file <none>
I20260810 19:32:14.062645 1506 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:14.062800 1506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:14.063122 1506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:14.066231 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "c1b13d316c914da3bc5c7044ebdc8d7b"
format_stamp: "Formatted at 2026-08-10 19:32:14 on dist-test-slave-tm1g"
server_key: "96204836ccfef1b4e19a40887a24ff7f"
server_key_iv: "40ccc0f038414c85fcafc97fd65624a9"
server_key_version: "encryptionkey@0"
I20260810 19:32:14.067113 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "c1b13d316c914da3bc5c7044ebdc8d7b"
format_stamp: "Formatted at 2026-08-10 19:32:14 on dist-test-slave-tm1g"
server_key: "96204836ccfef1b4e19a40887a24ff7f"
server_key_iv: "40ccc0f038414c85fcafc97fd65624a9"
server_key_version: "encryptionkey@0"
I20260810 19:32:14.072369 1506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:32:14.075953 1522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:14.077332 1506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:32:14.077534 1506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "c1b13d316c914da3bc5c7044ebdc8d7b"
format_stamp: "Formatted at 2026-08-10 19:32:14 on dist-test-slave-tm1g"
server_key: "96204836ccfef1b4e19a40887a24ff7f"
server_key_iv: "40ccc0f038414c85fcafc97fd65624a9"
server_key_version: "encryptionkey@0"
I20260810 19:32:14.077735 1506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:14.110553 1506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:14.115298 1506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:14.115620 1506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:14.135560 1506 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:34997
I20260810 19:32:14.135602 1577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:34997 every 8 connection(s)
I20260810 19:32:14.137037 1506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:14.138963 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 1506
I20260810 19:32:14.139214 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:14.140079 30630 external_mini_cluster.cc:1428] Setting key bc0a621ce6d4db9ecbb06aa2500ed555
I20260810 19:32:14.141832 1580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 19:32:14 dist-test-slave-tm1g krb5kdc[1491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:14.156001 1580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b: Bootstrap starting.
I20260810 19:32:14.160383 1580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:14.161790 1580 log.cc:824] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:14.164822 1584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:14.142079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.29.233.190:34997)
0810 19:32:14.142977 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:14.142988 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:14.143031 (+ 43us) server_negotiation.cc:407] Connection header received
0810 19:32:14.144623 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:14.144643 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:14.145024 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:14.145368 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:14.147252 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:14.148619 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:14.150431 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:14.151082 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:14.151214 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:14.155933 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:14.155945 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:14.155964 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:14.156035 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:14.159947 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:14.160797 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:14.160809 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:14.160824 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:14.161050 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:14.161415 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:14.161423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:14.161429 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:14.162210 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:14.162601 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:14.163075 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 19:32:14.163497 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":445,"threads_started":1}
I20260810 19:32:14.165419 1580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b: No bootstrap required, opened a new log
I20260810 19:32:14.169075 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:32:14.173324 1580 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34997 } }
I20260810 19:32:14.173699 1580 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:14.173810 1580 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b13d316c914da3bc5c7044ebdc8d7b, State: Initialized, Role: FOLLOWER
I20260810 19:32:14.174510 1580 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34997 } }
I20260810 19:32:14.174760 1580 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:14.174913 1580 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:14.175105 1580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:14.177196 1580 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34997 } }
I20260810 19:32:14.177695 1580 leader_election.cc:302] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1b13d316c914da3bc5c7044ebdc8d7b; no voters:
I20260810 19:32:14.178423 1580 leader_election.cc:288] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:14.178530 1588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:14.179901 1588 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [term 1 LEADER]: Becoming Leader. State: Replica: c1b13d316c914da3bc5c7044ebdc8d7b, State: Running, Role: LEADER
I20260810 19:32:14.180507 1588 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34997 } }
I20260810 19:32:14.180848 1580 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:14.183471 1591 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1b13d316c914da3bc5c7044ebdc8d7b. Latest consensus state: current_term: 1 leader_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34997 } } }
I20260810 19:32:14.183511 1589 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b13d316c914da3bc5c7044ebdc8d7b" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34997 } } }
I20260810 19:32:14.183892 1591 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:14.183947 1589 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:14.196751 1597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:14.202487 1597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:14.209192 1609 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 19:32:14.213455 1597 catalog_manager.cc:1347] Generated new cluster ID: b10afa999fc648d787301f4bc84e0be8
I20260810 19:32:14.213605 1597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:14.227315 1584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:14.217938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.29.233.190:34997)
0810 19:32:14.218130 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:14.218139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:14.218656 (+ 517us) server_negotiation.cc:407] Connection header received
0810 19:32:14.218828 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:14.218834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:14.218926 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:14.219047 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:14.220199 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:14.221385 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:14.223188 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:14.223904 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:14.224036 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:14.225211 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:14.225218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:14.225224 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:14.226157 (+ 933us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:14.226222 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:14.226270 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:14.226423 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:14.226605 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 19:32:14.226709 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 19:32:14.230472 1597 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:14.232236 1597 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:14.249037 1597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1b13d316c914da3bc5c7044ebdc8d7b: Generated new TSK 0
I20260810 19:32:14.250269 1597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:32:14.265899 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1506
I20260810 19:32:14.267156 1573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:14.359192 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 1506
2026-08-10T19:32:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2071 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:14 dist-test-slave-tm1g krb5kdc[1620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:14 dist-test-slave-tm1g krb5kdc[1620](info): set up 2 sockets
Aug 10 19:32:14 dist-test-slave-tm1g krb5kdc[1620](info): commencing operation
krb5kdc: starting...
W20260810 19:32:15.516350 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:15 dist-test-slave-tm1g krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:15.764299 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33381
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:16.119117 1633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:16.119436 1633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:16.119552 1633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:16.126896 1633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:16.127003 1633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:16.127051 1633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:16.127102 1633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:16.127147 1633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:16.137044 1633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:33381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:16.138502 1633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:16.139823 1633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:16.152127 1639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:16.152199 1640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:16.152793 1643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:16.152875 1633 server_base.cc:1034] running on GCE node
I20260810 19:32:16.153750 1633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:16.155591 1633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:16.156849 1633 hybrid_clock.cc:648] HybridClock initialized: now 1786390336156805 us; error 43 us; skew 500 ppm
I20260810 19:32:16.157224 1633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:16 dist-test-slave-tm1g krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390336, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:16.160503 1633 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:16.162066 1633 webserver.cc:460] Webserver started at http://127.29.233.190:46845/ using document root <none> and password file <none>
I20260810 19:32:16.162882 1633 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:16.163043 1633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:16.163379 1633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:16.166666 1633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "e8e13211dca24f47b343db271085481d"
format_stamp: "Formatted at 2026-08-10 19:32:16 on dist-test-slave-tm1g"
server_key: "13208ea7f66bfb7c1668a2ab4c7ef257"
server_key_iv: "99e5cceede557fe74a35db967ecf4d85"
server_key_version: "encryptionkey@0"
I20260810 19:32:16.167490 1633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "e8e13211dca24f47b343db271085481d"
format_stamp: "Formatted at 2026-08-10 19:32:16 on dist-test-slave-tm1g"
server_key: "13208ea7f66bfb7c1668a2ab4c7ef257"
server_key_iv: "99e5cceede557fe74a35db967ecf4d85"
server_key_version: "encryptionkey@0"
I20260810 19:32:16.173039 1633 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 19:32:16.176785 1651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:16.178194 1633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:16.178433 1633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "e8e13211dca24f47b343db271085481d"
format_stamp: "Formatted at 2026-08-10 19:32:16 on dist-test-slave-tm1g"
server_key: "13208ea7f66bfb7c1668a2ab4c7ef257"
server_key_iv: "99e5cceede557fe74a35db967ecf4d85"
server_key_version: "encryptionkey@0"
I20260810 19:32:16.178664 1633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:16.218916 1633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:16.223080 1633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:16.223392 1633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:16.242390 1633 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:33381
I20260810 19:32:16.242503 1707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:33381 every 8 connection(s)
I20260810 19:32:16.243873 1633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:16.247951 1708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:16.248485 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 1633
I20260810 19:32:16.248720 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:16.249373 30630 external_mini_cluster.cc:1428] Setting key 390aa48ddc41d1563c4288816654d87d
I20260810 19:32:16.262410 1708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d: Bootstrap starting.
Aug 10 19:32:16 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:16.265655 1708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:16.267114 1708 log.cc:824] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:16.270524 1708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d: No bootstrap required, opened a new log
I20260810 19:32:16.274070 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:16.251369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.29.233.190:33381)
0810 19:32:16.252169 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:16.252181 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:16.252227 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:32:16.253971 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:16.253993 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:16.254435 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:16.254919 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:16.257030 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:16.259130 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:16.260380 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:16.261432 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:16.261607 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:16.265565 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:16.265576 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:16.265596 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:16.265651 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:16.269515 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:16.270224 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:16.270232 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:16.270239 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:16.270476 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:16.270896 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:16.270900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:16.270905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:16.271634 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:16.271947 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:16.272230 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 19:32:16.272682 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":355,"threads_started":1}
I20260810 19:32:16.279651 1708 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e13211dca24f47b343db271085481d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33381 } }
I20260810 19:32:16.280057 1708 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:16.280187 1708 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e13211dca24f47b343db271085481d, State: Initialized, Role: FOLLOWER
I20260810 19:32:16.280819 1708 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e13211dca24f47b343db271085481d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33381 } }
I20260810 19:32:16.281023 1708 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:16.281127 1708 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:16.281248 1708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:16.284036 1708 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e13211dca24f47b343db271085481d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33381 } }
I20260810 19:32:16.284508 1708 leader_election.cc:302] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e13211dca24f47b343db271085481d; no voters:
I20260810 19:32:16.285030 1708 leader_election.cc:288] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:16.285185 1714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:16.285884 1714 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [term 1 LEADER]: Becoming Leader. State: Replica: e8e13211dca24f47b343db271085481d, State: Running, Role: LEADER
I20260810 19:32:16.286482 1714 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e13211dca24f47b343db271085481d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33381 } }
I20260810 19:32:16.287189 1708 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:16.289350 1716 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e13211dca24f47b343db271085481d. Latest consensus state: current_term: 1 leader_uuid: "e8e13211dca24f47b343db271085481d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e13211dca24f47b343db271085481d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33381 } } }
I20260810 19:32:16.289664 1716 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:16.290621 1719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:16.290499 1715 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8e13211dca24f47b343db271085481d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e13211dca24f47b343db271085481d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33381 } } }
I20260810 19:32:16.290845 1715 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:16.296398 1719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:16.305462 1719 catalog_manager.cc:1347] Generated new cluster ID: 4da8ecbe14774ecf8c1c7683eb5f38ca
I20260810 19:32:16.305600 1719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:16.324920 1719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:16.326593 1719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:16.338112 1719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8e13211dca24f47b343db271085481d: Generated new TSK 0
I20260810 19:32:16.339033 1719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:32:16.402053 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33381
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:16.741365 1739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:16.741698 1739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:16.741895 1739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:16.749253 1739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:16.749410 1739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:16.749612 1739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:16.762187 1739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:16.763800 1739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:16.765185 1739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:16.779981 1746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:16.780035 1747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:16.779933 1750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:16.781220 1739 server_base.cc:1034] running on GCE node
I20260810 19:32:16.781942 1739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:16.783083 1739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:16.784289 1739 hybrid_clock.cc:648] HybridClock initialized: now 1786390336784247 us; error 38 us; skew 500 ppm
I20260810 19:32:16.784619 1739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:16 dist-test-slave-tm1g krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390336, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:16.788316 1739 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:32:16.789885 1739 webserver.cc:460] Webserver started at http://127.29.233.129:37219/ using document root <none> and password file <none>
I20260810 19:32:16.790665 1739 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:16.790773 1739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:16.791085 1739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:16.794397 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "a3211feb0e574406adb502178cb8230d"
format_stamp: "Formatted at 2026-08-10 19:32:16 on dist-test-slave-tm1g"
server_key: "1e2054e061e780ba1805c69c2afcb534"
server_key_iv: "ecfa7fa9d2b3ccbe2053dd9558afa1db"
server_key_version: "encryptionkey@0"
I20260810 19:32:16.795220 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "a3211feb0e574406adb502178cb8230d"
format_stamp: "Formatted at 2026-08-10 19:32:16 on dist-test-slave-tm1g"
server_key: "1e2054e061e780ba1805c69c2afcb534"
server_key_iv: "ecfa7fa9d2b3ccbe2053dd9558afa1db"
server_key_version: "encryptionkey@0"
I20260810 19:32:16.800462 1739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 19:32:16.803953 1758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:16.805259 1739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:32:16.805442 1739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "a3211feb0e574406adb502178cb8230d"
format_stamp: "Formatted at 2026-08-10 19:32:16 on dist-test-slave-tm1g"
server_key: "1e2054e061e780ba1805c69c2afcb534"
server_key_iv: "ecfa7fa9d2b3ccbe2053dd9558afa1db"
server_key_version: "encryptionkey@0"
I20260810 19:32:16.805647 1739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:16.829041 1739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:16.833309 1739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:16.833633 1739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:16.834744 1739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:16.836244 1739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:16.836319 1739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:16.836483 1739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:16.836549 1739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:16.871831 1739 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:34665
I20260810 19:32:16.871877 1879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:34665 every 8 connection(s)
I20260810 19:32:16.873186 1739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:16.876788 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 1739
I20260810 19:32:16.877027 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:16.877882 30630 external_mini_cluster.cc:1428] Setting key 340a7eca4bcdaa90322fecb600d69f1e
Aug 10 19:32:16 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390336, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:16.894761 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:16.875707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:39237 (local address 127.29.233.190:33381)
0810 19:32:16.876172 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:16.876177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:16.877186 (+ 1009us) server_negotiation.cc:407] Connection header received
0810 19:32:16.879554 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:16.879558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:16.879629 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:16.879723 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:16.881291 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:16.882615 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:16.884286 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:16.885022 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:16.885168 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:16.888728 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:16.888736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:16.888741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:16.888784 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:16.891402 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:16.892322 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:16.892326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:16.892328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:16.892442 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:16.892938 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:16.892941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:16.892943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:16.893256 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:16.893369 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:16.894267 (+ 898us) server_negotiation.cc:299] Negotiation successful
0810 19:32:16.894505 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":254,"threads_started":1}
I20260810 19:32:16.896353 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:16.876133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33381 (local address 127.29.233.129:39237)
0810 19:32:16.877011 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:16.877069 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:16.879268 (+ 2199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:16.880021 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:16.880032 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:16.880570 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:16.881112 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:16.881128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:16.882766 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:16.882769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:16.884128 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:16.884139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:16.885230 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:16.885235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:16.885436 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:16.886150 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:16.886171 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:16.888516 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:16.891557 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:16.891568 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:16.891588 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:16.892208 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:16.892559 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:16.892567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:16.892573 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:16.892827 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:16.893392 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:16.893403 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:16.893861 (+ 458us) client_negotiation.cc:769] Sending connection context
0810 19:32:16.894176 (+ 315us) client_negotiation.cc:240] Negotiation successful
0810 19:32:16.894852 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":338,"threads_started":1}
I20260810 19:32:16.897997 1880 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33381
I20260810 19:32:16.898370 1880 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:16.899346 1880 heartbeater.cc:511] Master 127.29.233.190:33381 requested a full tablet report, sending...
I20260810 19:32:16.901969 1671 ts_manager.cc:194] Registered new tserver with Master: a3211feb0e574406adb502178cb8230d (127.29.233.129:34665)
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
I20260810 19:32:16.903668 1671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:39237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:32:16.938860 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33381
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:17.276934 1887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:17.277256 1887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:17.277427 1887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:17.284945 1887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:17.285105 1887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:17.285302 1887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:17.295337 1887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:17.296666 1887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:17.298043 1887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:17.311793 1893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:17.311834 1896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:17.311862 1894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:17.312481 1887 server_base.cc:1034] running on GCE node
I20260810 19:32:17.313109 1887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:17.314080 1887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:17.315308 1887 hybrid_clock.cc:648] HybridClock initialized: now 1786390337315246 us; error 74 us; skew 500 ppm
I20260810 19:32:17.315661 1887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:17 dist-test-slave-tm1g krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:17.319151 1887 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:32:17.320775 1887 webserver.cc:460] Webserver started at http://127.29.233.130:41301/ using document root <none> and password file <none>
I20260810 19:32:17.321519 1887 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:17.321673 1887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:17.322099 1887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:17.325199 1887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "419b7da46b4c458ea27bca99720627b3"
format_stamp: "Formatted at 2026-08-10 19:32:17 on dist-test-slave-tm1g"
server_key: "73e2c8baed241a4d55b6973245f3b1e5"
server_key_iv: "81910501b055c91629d89b80a65f54d9"
server_key_version: "encryptionkey@0"
I20260810 19:32:17.326036 1887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "419b7da46b4c458ea27bca99720627b3"
format_stamp: "Formatted at 2026-08-10 19:32:17 on dist-test-slave-tm1g"
server_key: "73e2c8baed241a4d55b6973245f3b1e5"
server_key_iv: "81910501b055c91629d89b80a65f54d9"
server_key_version: "encryptionkey@0"
I20260810 19:32:17.331410 1887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 19:32:17.335044 1904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:17.336551 1887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:17.336750 1887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "419b7da46b4c458ea27bca99720627b3"
format_stamp: "Formatted at 2026-08-10 19:32:17 on dist-test-slave-tm1g"
server_key: "73e2c8baed241a4d55b6973245f3b1e5"
server_key_iv: "81910501b055c91629d89b80a65f54d9"
server_key_version: "encryptionkey@0"
I20260810 19:32:17.336958 1887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:17.359833 1887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:17.364274 1887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:17.364552 1887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:17.365551 1887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:17.366909 1887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:17.366986 1887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:17.367084 1887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:17.367129 1887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:17.400369 1887 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:43215
I20260810 19:32:17.400416 2028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:43215 every 8 connection(s)
I20260810 19:32:17.401748 1887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:17.402510 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 1887
I20260810 19:32:17.402732 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:17.403337 30630 external_mini_cluster.cc:1428] Setting key 59c8e290c70e30677f9cbd186fd99bcf
Aug 10 19:32:17 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:17.423233 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:17.404733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:37637 (local address 127.29.233.190:33381)
0810 19:32:17.405232 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:17.405239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:17.406485 (+ 1246us) server_negotiation.cc:407] Connection header received
0810 19:32:17.408326 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:17.408331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:17.408421 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:17.408545 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:17.410131 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.410993 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.412535 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.412970 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.413112 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:17.417157 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:17.417162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:17.417165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:17.417200 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:17.420006 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:17.420772 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:17.420778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:17.420781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:17.420913 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:17.421371 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:17.421377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:17.421380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:17.421656 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:17.421817 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:17.422581 (+ 764us) server_negotiation.cc:299] Negotiation successful
0810 19:32:17.422814 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":259,"threads_started":1}
I20260810 19:32:17.424276 2032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:17.405494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33381 (local address 127.29.233.130:37637)
0810 19:32:17.406279 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:17.406383 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:17.408046 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:17.408810 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:17.408819 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:17.409326 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:17.409962 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:17.409979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.411141 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.411147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:17.412377 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:17.412389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.413218 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.413225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:17.413465 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:17.414182 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:17.414224 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:17.416901 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:17.420178 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:17.420187 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:17.420204 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:17.420647 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:17.421036 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:17.421041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:17.421045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:17.421258 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:17.421789 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:17.421797 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:17.422155 (+ 358us) client_negotiation.cc:769] Sending connection context
0810 19:32:17.422500 (+ 345us) client_negotiation.cc:240] Negotiation successful
0810 19:32:17.423069 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":323,"threads_started":1}
I20260810 19:32:17.425863 2029 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33381
I20260810 19:32:17.426190 2029 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
I20260810 19:32:17.427011 2029 heartbeater.cc:511] Master 127.29.233.190:33381 requested a full tablet report, sending...
I20260810 19:32:17.428618 1671 ts_manager.cc:194] Registered new tserver with Master: 419b7da46b4c458ea27bca99720627b3 (127.29.233.130:43215)
I20260810 19:32:17.429741 1671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:37637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:32:17.458820 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33381
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:17.804755 2036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:17.805044 2036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:17.805142 2036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:17.812625 2036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:17.812748 2036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:17.812866 2036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:17.822852 2036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:17.824218 2036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:17.825575 2036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:17.839366 2041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:17.839342 2042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:17.839457 2045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:17.840178 2036 server_base.cc:1034] running on GCE node
I20260810 19:32:17.840816 2036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:17.841964 2036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:17.843225 2036 hybrid_clock.cc:648] HybridClock initialized: now 1786390337843123 us; error 90 us; skew 500 ppm
I20260810 19:32:17.843573 2036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:17 dist-test-slave-tm1g krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:17.846995 2036 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:32:17.848506 2036 webserver.cc:460] Webserver started at http://127.29.233.131:33409/ using document root <none> and password file <none>
I20260810 19:32:17.849254 2036 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:17.849408 2036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:17.849745 2036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:17.852914 2036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "d755439303284ec190e308be4e411ec9"
format_stamp: "Formatted at 2026-08-10 19:32:17 on dist-test-slave-tm1g"
server_key: "83c01bd4dd0c93d065a40fbf8d7885bb"
server_key_iv: "c2423610f66c002e0ef11a7eba0cfd54"
server_key_version: "encryptionkey@0"
I20260810 19:32:17.853756 2036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "d755439303284ec190e308be4e411ec9"
format_stamp: "Formatted at 2026-08-10 19:32:17 on dist-test-slave-tm1g"
server_key: "83c01bd4dd0c93d065a40fbf8d7885bb"
server_key_iv: "c2423610f66c002e0ef11a7eba0cfd54"
server_key_version: "encryptionkey@0"
I20260810 19:32:17.859179 2036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 19:32:17.862679 2054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:17.864004 2036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:32:17.864202 2036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "d755439303284ec190e308be4e411ec9"
format_stamp: "Formatted at 2026-08-10 19:32:17 on dist-test-slave-tm1g"
server_key: "83c01bd4dd0c93d065a40fbf8d7885bb"
server_key_iv: "c2423610f66c002e0ef11a7eba0cfd54"
server_key_version: "encryptionkey@0"
I20260810 19:32:17.864404 2036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:17.887187 2036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:17.891546 2036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:17.891873 2036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:17.892879 2036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:17.894351 2036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:17.894461 2036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:17.894623 2036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:17.894697 2036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:17.907001 1880 heartbeater.cc:503] Master 127.29.233.190:33381 was elected leader, sending a full tablet report...
I20260810 19:32:17.930553 2036 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:44651
I20260810 19:32:17.930668 2181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:44651 every 8 connection(s)
I20260810 19:32:17.932027 2036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:17.932615 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 2036
I20260810 19:32:17.932844 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:17.933506 30630 external_mini_cluster.cc:1428] Setting key a9ea31fef726b9fa4f8e2595a752af91
Aug 10 19:32:17 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:17.953889 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:17.934798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:36967 (local address 127.29.233.190:33381)
0810 19:32:17.938637 (+ 3839us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:17.938641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:17.938658 (+ 17us) server_negotiation.cc:407] Connection header received
0810 19:32:17.939470 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:17.939474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:17.939555 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:17.939642 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:17.941078 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.941994 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.943739 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.944131 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.944311 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:17.947952 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:17.947958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:17.947963 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:17.948003 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:17.951061 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:17.951787 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:17.951791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:17.951793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:17.951941 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:17.952346 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:17.952352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:17.952355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:17.952632 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:17.952753 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:17.953427 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 19:32:17.953599 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3745,"thread_start_us":3615,"threads_started":1}
I20260810 19:32:17.954938 2185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:17.935438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33381 (local address 127.29.233.131:36967)
0810 19:32:17.936740 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:17.936813 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:17.939223 (+ 2410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:17.939813 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:17.939820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:17.940333 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:17.940885 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:17.940906 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.942185 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.942196 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:17.943585 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:17.943598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:17.944405 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:17.944416 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:17.944665 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:17.945373 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:17.945400 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:17.947710 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:17.951202 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:17.951210 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:17.951222 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:17.951665 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:17.952039 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:17.952042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:17.952044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:17.952222 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:17.952745 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:17.952751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:17.953069 (+ 318us) client_negotiation.cc:769] Sending connection context
0810 19:32:17.953301 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 19:32:17.953697 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":676,"threads_started":1}
I20260810 19:32:17.956447 2182 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33381
I20260810 19:32:17.956777 2182 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:17.957515 2182 heartbeater.cc:511] Master 127.29.233.190:33381 requested a full tablet report, sending...
I20260810 19:32:17.959136 1671 ts_manager.cc:194] Registered new tserver with Master: d755439303284ec190e308be4e411ec9 (127.29.233.131:44651)
I20260810 19:32:17.959990 1671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:36967
I20260810 19:32:17.962292 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 19:32:17 dist-test-slave-tm1g krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 19:32:18 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:18.022046 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:18.006585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.29.233.190:33381)
0810 19:32:18.006829 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:18.006834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:18.007278 (+ 444us) server_negotiation.cc:407] Connection header received
0810 19:32:18.007432 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:18.007436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:18.007524 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:18.007638 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:18.009036 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.009895 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.011388 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.011834 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.011935 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:18.016235 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:18.016241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:18.016244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:18.016283 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:18.019500 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.020177 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.020181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.020184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.020311 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.020670 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.020673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.020675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.021007 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:18.021185 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:18.021615 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 19:32:18.021803 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260810 19:32:18.022780 1671 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34260
I20260810 19:32:18.024668 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1739
Aug 10 19:32:18 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:18.041450 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:18.025821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.29.233.129:34665)
0810 19:32:18.026531 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:18.026539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:18.026563 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:32:18.026618 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:18.026624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:18.026837 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:18.026971 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:18.028108 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.029134 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.030500 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.031101 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.031208 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:18.035524 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:18.035537 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:18.035552 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:18.035599 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:18.038608 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.039332 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.039340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.039343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.039525 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.039904 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.039908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.039910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.040282 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:18.040521 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:18.040903 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 19:32:18.041115 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":363,"threads_started":1}
W20260810 19:32:18.043138 1875 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46860
W20260810 19:32:18.044445 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1739: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:32:18.044698 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 1739
I20260810 19:32:18.063812 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1887
Aug 10 19:32:18 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:32:18.080973 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:18.064844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.29.233.130:43215)
0810 19:32:18.065487 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:18.065494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:18.065521 (+ 27us) server_negotiation.cc:407] Connection header received
0810 19:32:18.065585 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:18.065592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:18.065831 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:18.065967 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:18.067489 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.068718 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.070168 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.070837 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.070953 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:18.075162 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:18.075173 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:18.075189 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:18.075232 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:18.078180 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.078920 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.078927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.078929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.079071 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.079452 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.079456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.079458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.079844 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:18.080082 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:18.080338 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 19:32:18.080576 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":294,"threads_started":1}
W20260810 19:32:18.082356 2024 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59186
W20260810 19:32:18.083593 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1887: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:32:18.083768 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 1887
I20260810 19:32:18.105173 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2036
Aug 10 19:32:18 dist-test-slave-tm1g krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390337, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:32:18.122198 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:18.106261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.29.233.131:44651)
0810 19:32:18.106951 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:18.106959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:18.106993 (+ 34us) server_negotiation.cc:407] Connection header received
0810 19:32:18.107058 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:18.107065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:18.107284 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:18.107415 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:18.108858 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.109934 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.111263 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:18.111823 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:18.111924 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:18.116298 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:18.116310 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:18.116324 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:18.116366 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:18.119451 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.120141 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.120148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.120151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.120301 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:18.120708 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:18.120713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:18.120715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:18.121106 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:18.121367 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:18.121684 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 19:32:18.121907 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":330,"threads_started":1}
W20260810 19:32:18.123642 2175 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42412
W20260810 19:32:18.124883 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2036: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:32:18.125106 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 2036
I20260810 19:32:18.144106 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 1633
I20260810 19:32:18.145503 1703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:18.252434 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 1633
2026-08-10T19:32:18Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3901 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:18 dist-test-slave-tm1g krb5kdc[2201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:18 dist-test-slave-tm1g krb5kdc[2201](info): set up 2 sockets
Aug 10 19:32:18 dist-test-slave-tm1g krb5kdc[2201](info): commencing operation
krb5kdc: starting...
W20260810 19:32:19.415297 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:19 dist-test-slave-tm1g krb5kdc[2201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:19.667060 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42121
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:42121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:20.009869 2215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:20.010176 2215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:20.010404 2215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:20.017695 2215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:20.017812 2215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:20.017900 2215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:20.018010 2215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:20.018072 2215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:20.028272 2215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:42121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:20.029747 2215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:20.031162 2215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:20.043581 2223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:20.043619 2220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:20.043642 2221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:20.044478 2215 server_base.cc:1034] running on GCE node
I20260810 19:32:20.045361 2215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:20.046917 2215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:20.048141 2215 hybrid_clock.cc:648] HybridClock initialized: now 1786390340048085 us; error 38 us; skew 500 ppm
I20260810 19:32:20.048544 2215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:20 dist-test-slave-tm1g krb5kdc[2201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390340, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:20.057813 2215 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:20.059676 2215 webserver.cc:460] Webserver started at http://127.29.233.190:45665/ using document root <none> and password file <none>
I20260810 19:32:20.060495 2215 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:20.060631 2215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:20.060992 2215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:20.064239 2215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "b84ba8d5adf84554b27107a3190b8c03"
format_stamp: "Formatted at 2026-08-10 19:32:20 on dist-test-slave-tm1g"
server_key: "57500d48a55c63c028c4f9f65ae96436"
server_key_iv: "64c85c8dd2bbf4d6c263089d6b736214"
server_key_version: "encryptionkey@0"
I20260810 19:32:20.065093 2215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "b84ba8d5adf84554b27107a3190b8c03"
format_stamp: "Formatted at 2026-08-10 19:32:20 on dist-test-slave-tm1g"
server_key: "57500d48a55c63c028c4f9f65ae96436"
server_key_iv: "64c85c8dd2bbf4d6c263089d6b736214"
server_key_version: "encryptionkey@0"
I20260810 19:32:20.070659 2215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 19:32:20.074257 2234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:20.075711 2215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:20.075927 2215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "b84ba8d5adf84554b27107a3190b8c03"
format_stamp: "Formatted at 2026-08-10 19:32:20 on dist-test-slave-tm1g"
server_key: "57500d48a55c63c028c4f9f65ae96436"
server_key_iv: "64c85c8dd2bbf4d6c263089d6b736214"
server_key_version: "encryptionkey@0"
I20260810 19:32:20.076143 2215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:20.089123 2215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:20.093560 2215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:20.093803 2215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:20.113394 2215 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:42121
I20260810 19:32:20.113416 2287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:42121 every 8 connection(s)
I20260810 19:32:20.114952 2215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:20.118964 2288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:20.121523 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 2215
I20260810 19:32:20.121769 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:20.122592 30630 external_mini_cluster.cc:1428] Setting key 7d7a27628f7649ea02eed3dc70c34e1c
I20260810 19:32:20.132344 2288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03: Bootstrap starting.
I20260810 19:32:20.136413 2288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:20.137773 2288 log.cc:824] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03: Log is configured to *not* fsync() on all Append() calls
Aug 10 19:32:20 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:20.141435 2288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03: No bootstrap required, opened a new log
I20260810 19:32:20.148689 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:20.124870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.29.233.190:42121)
0810 19:32:20.125735 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:20.125748 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:20.125789 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:32:20.127547 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:20.127571 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:20.127970 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:20.128325 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:20.130209 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:20.131604 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:20.133407 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:20.134103 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:20.134235 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:20.139503 (+ 5268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:20.139515 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:20.139534 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:20.139582 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:20.143522 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:20.144256 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:20.144267 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:20.144281 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:20.144522 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:20.144942 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:20.144948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:20.144951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:20.145784 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:20.146120 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:20.146493 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 19:32:20.147104 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":403,"threads_started":1}
I20260810 19:32:20.151894 2288 raft_consensus.cc:348] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84ba8d5adf84554b27107a3190b8c03" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42121 } }
I20260810 19:32:20.152246 2288 raft_consensus.cc:374] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:20.152325 2288 raft_consensus.cc:729] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84ba8d5adf84554b27107a3190b8c03, State: Initialized, Role: FOLLOWER
I20260810 19:32:20.152798 2288 consensus_queue.cc:260] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84ba8d5adf84554b27107a3190b8c03" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42121 } }
I20260810 19:32:20.153009 2288 raft_consensus.cc:388] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:20.153102 2288 raft_consensus.cc:482] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:20.153275 2288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:20.155783 2288 raft_consensus.cc:504] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84ba8d5adf84554b27107a3190b8c03" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42121 } }
I20260810 19:32:20.156318 2288 leader_election.cc:302] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b84ba8d5adf84554b27107a3190b8c03; no voters:
I20260810 19:32:20.156965 2288 leader_election.cc:288] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:20.157133 2293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:20.157886 2293 raft_consensus.cc:686] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [term 1 LEADER]: Becoming Leader. State: Replica: b84ba8d5adf84554b27107a3190b8c03, State: Running, Role: LEADER
I20260810 19:32:20.158488 2293 consensus_queue.cc:237] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84ba8d5adf84554b27107a3190b8c03" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42121 } }
I20260810 19:32:20.159160 2288 sys_catalog.cc:564] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:20.161624 2294 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b84ba8d5adf84554b27107a3190b8c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84ba8d5adf84554b27107a3190b8c03" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42121 } } }
I20260810 19:32:20.162092 2294 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:20.162426 2296 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b84ba8d5adf84554b27107a3190b8c03. Latest consensus state: current_term: 1 leader_uuid: "b84ba8d5adf84554b27107a3190b8c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84ba8d5adf84554b27107a3190b8c03" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42121 } } }
I20260810 19:32:20.162905 2296 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:20.164693 2300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:20.169780 2300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:20.178728 2300 catalog_manager.cc:1347] Generated new cluster ID: ca9d9920597f4deea3df7bad0e51b34f
I20260810 19:32:20.178889 2300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:20.211514 2300 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:20.213585 2300 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:20.223536 2300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b84ba8d5adf84554b27107a3190b8c03: Generated new TSK 0
I20260810 19:32:20.224700 2300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:32:20.290773 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42121
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:20.642835 2316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:20.643116 2316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:20.643210 2316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:20.650616 2316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:20.650785 2316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:20.650949 2316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:20.660962 2316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:20.662282 2316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:20.663697 2316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:20.678350 2327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:20.678426 2324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:20.678454 2325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:20.679389 2316 server_base.cc:1034] running on GCE node
I20260810 19:32:20.680079 2316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:20.681202 2316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:20.682463 2316 hybrid_clock.cc:648] HybridClock initialized: now 1786390340682413 us; error 44 us; skew 500 ppm
I20260810 19:32:20.682838 2316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:20 dist-test-slave-tm1g krb5kdc[2201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390340, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:20.686082 2316 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:32:20.687610 2316 webserver.cc:460] Webserver started at http://127.29.233.129:42591/ using document root <none> and password file <none>
I20260810 19:32:20.688267 2316 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:20.688380 2316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:20.688635 2316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:20.691601 2316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "1ca80776b8764585bd651ad194e3e4ed"
format_stamp: "Formatted at 2026-08-10 19:32:20 on dist-test-slave-tm1g"
server_key: "5473310da8232966bb0db2dddfab6ca1"
server_key_iv: "49f875879e16d6c1401bd55d25f4b360"
server_key_version: "encryptionkey@0"
I20260810 19:32:20.692369 2316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "1ca80776b8764585bd651ad194e3e4ed"
format_stamp: "Formatted at 2026-08-10 19:32:20 on dist-test-slave-tm1g"
server_key: "5473310da8232966bb0db2dddfab6ca1"
server_key_iv: "49f875879e16d6c1401bd55d25f4b360"
server_key_version: "encryptionkey@0"
I20260810 19:32:20.697629 2316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:32:20.701453 2336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:20.703047 2316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:20.703186 2316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "1ca80776b8764585bd651ad194e3e4ed"
format_stamp: "Formatted at 2026-08-10 19:32:20 on dist-test-slave-tm1g"
server_key: "5473310da8232966bb0db2dddfab6ca1"
server_key_iv: "49f875879e16d6c1401bd55d25f4b360"
server_key_version: "encryptionkey@0"
I20260810 19:32:20.703374 2316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:20.730863 2316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:20.735311 2316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:20.735576 2316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:20.736485 2316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:20.737802 2316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:20.737879 2316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:20.737963 2316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:20.738008 2316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:20.773491 2316 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:46759
I20260810 19:32:20.773587 2454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:46759 every 8 connection(s)
I20260810 19:32:20.774931 2316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:20.777091 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 2316
I20260810 19:32:20.777318 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:20.778106 30630 external_mini_cluster.cc:1428] Setting key 7e591b278209034c912798f7f581468b
Aug 10 19:32:20 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390340, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:20.798918 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:20.777775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:55087 (local address 127.29.233.190:42121)
0810 19:32:20.778321 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:20.778329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:20.779401 (+ 1072us) server_negotiation.cc:407] Connection header received
0810 19:32:20.781982 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:20.781987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:20.782054 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:20.782153 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:20.784296 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:20.785069 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:20.787562 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:20.787972 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:20.788081 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:20.791856 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:20.791865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:20.791873 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:20.791921 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:20.795453 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:20.796174 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:20.796182 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:20.796189 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:20.796342 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:20.796771 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:20.796778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:20.796784 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:20.797214 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:20.797391 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:20.798085 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 19:32:20.798384 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":239,"threads_started":1}
I20260810 19:32:20.799616 2459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:20.778441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42121 (local address 127.29.233.129:55087)
0810 19:32:20.779234 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:20.779286 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:20.781666 (+ 2380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:20.782450 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:20.782465 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:20.783270 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:20.784090 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:20.784119 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:20.785220 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:20.785228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:20.787403 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:20.787428 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:20.788208 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:20.788217 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:20.788558 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:20.789306 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:20.789340 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:20.791597 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:20.795596 (+ 3999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:20.795603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:20.795616 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:20.796032 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:20.796451 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:20.796454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:20.796456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:20.796654 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:20.797351 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:20.797357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:20.797687 (+ 330us) client_negotiation.cc:769] Sending connection context
0810 19:32:20.797913 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 19:32:20.798383 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":299,"threads_started":1}
I20260810 19:32:20.801373 2455 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42121
I20260810 19:32:20.801755 2455 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:20.802762 2455 heartbeater.cc:511] Master 127.29.233.190:42121 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
I20260810 19:32:20.805210 2251 ts_manager.cc:194] Registered new tserver with Master: 1ca80776b8764585bd651ad194e3e4ed (127.29.233.129:46759)
I20260810 19:32:20.807051 2251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:55087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:32:20.838387 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42121
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:21.171923 2463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:21.172221 2463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:21.172415 2463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:21.179544 2463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:21.179693 2463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:21.179829 2463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:21.189357 2463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:21.190657 2463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:21.191987 2463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:21.205713 2469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:21.205668 2470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:21.205744 2472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:21.206544 2463 server_base.cc:1034] running on GCE node
I20260810 19:32:21.207185 2463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:21.208212 2463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:21.209472 2463 hybrid_clock.cc:648] HybridClock initialized: now 1786390341209435 us; error 44 us; skew 500 ppm
I20260810 19:32:21.209823 2463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:21 dist-test-slave-tm1g krb5kdc[2201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:21.212988 2463 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:32:21.214586 2463 webserver.cc:460] Webserver started at http://127.29.233.130:46833/ using document root <none> and password file <none>
I20260810 19:32:21.215306 2463 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:21.215456 2463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:21.215781 2463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:21.218819 2463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "a989aa363a874378a6bfbf9702cdb100"
format_stamp: "Formatted at 2026-08-10 19:32:21 on dist-test-slave-tm1g"
server_key: "b1045b5ae18cf9ae1f3bb002a428c9bb"
server_key_iv: "7e4928ce089e5c912d52ae9bc73ab36e"
server_key_version: "encryptionkey@0"
I20260810 19:32:21.219666 2463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "a989aa363a874378a6bfbf9702cdb100"
format_stamp: "Formatted at 2026-08-10 19:32:21 on dist-test-slave-tm1g"
server_key: "b1045b5ae18cf9ae1f3bb002a428c9bb"
server_key_iv: "7e4928ce089e5c912d52ae9bc73ab36e"
server_key_version: "encryptionkey@0"
I20260810 19:32:21.224885 2463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:32:21.228434 2480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:21.230119 2463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 19:32:21.230252 2463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "a989aa363a874378a6bfbf9702cdb100"
format_stamp: "Formatted at 2026-08-10 19:32:21 on dist-test-slave-tm1g"
server_key: "b1045b5ae18cf9ae1f3bb002a428c9bb"
server_key_iv: "7e4928ce089e5c912d52ae9bc73ab36e"
server_key_version: "encryptionkey@0"
I20260810 19:32:21.230520 2463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:21.242851 2463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:21.246742 2463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:21.246975 2463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:21.247807 2463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:21.249046 2463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:21.249119 2463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:21.249202 2463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:21.249246 2463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:21.285183 2463 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:37845
I20260810 19:32:21.285219 2601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:37845 every 8 connection(s)
I20260810 19:32:21.286586 2463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:21.290669 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 2463
I20260810 19:32:21.290887 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:21.291328 30630 external_mini_cluster.cc:1428] Setting key 9b2e7170cba6d38435119a288e02e391
Aug 10 19:32:21 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:21.308276 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:21.289161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:50153 (local address 127.29.233.190:42121)
0810 19:32:21.289354 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:21.289359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:21.290477 (+ 1118us) server_negotiation.cc:407] Connection header received
0810 19:32:21.292633 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:21.292639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:21.292735 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:21.292859 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:21.294380 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.295099 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.296677 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.297113 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.297251 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:21.301005 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:21.301010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:21.301013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:21.301047 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:21.304653 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.305425 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.305433 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.305439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.305611 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.306048 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.306055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.306062 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.306492 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:21.306668 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:21.307475 (+ 807us) server_negotiation.cc:299] Negotiation successful
0810 19:32:21.307757 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:32:21.309044 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:21.289550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42121 (local address 127.29.233.130:50153)
0810 19:32:21.290261 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:21.290352 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:21.292367 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:21.293091 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:21.293100 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:21.293619 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:21.294185 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:21.294202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.295269 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.295273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:21.296523 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:21.296541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.297366 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.297375 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:21.297599 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:21.298296 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:21.298403 (+ 107us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:21.300755 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:21.304804 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:21.304810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:21.304825 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:21.305262 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:21.305730 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:21.305733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:21.305735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:21.305917 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:21.306631 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:21.306645 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:21.307022 (+ 377us) client_negotiation.cc:769] Sending connection context
0810 19:32:21.307319 (+ 297us) client_negotiation.cc:240] Negotiation successful
0810 19:32:21.307792 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":275,"threads_started":1}
I20260810 19:32:21.310659 2602 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42121
I20260810 19:32:21.311004 2602 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:21.311733 2602 heartbeater.cc:511] Master 127.29.233.190:42121 requested a full tablet report, sending...
I20260810 19:32:21.313254 2251 ts_manager.cc:194] Registered new tserver with Master: a989aa363a874378a6bfbf9702cdb100 (127.29.233.130:37845)
I20260810 19:32:21.314080 2251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:50153
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:32:21.350209 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42121
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:21.684794 2608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:21.685086 2608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:21.685184 2608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:21.692476 2608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:21.692584 2608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:21.692700 2608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:21.702411 2608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:21.703739 2608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:21.705045 2608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:21.720376 2614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:21.720330 2617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:21.720420 2615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:21.721139 2608 server_base.cc:1034] running on GCE node
I20260810 19:32:21.721733 2608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:21.722822 2608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:21.724042 2608 hybrid_clock.cc:648] HybridClock initialized: now 1786390341723990 us; error 48 us; skew 500 ppm
I20260810 19:32:21.724378 2608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:21 dist-test-slave-tm1g krb5kdc[2201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:21.728013 2608 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:32:21.729565 2608 webserver.cc:460] Webserver started at http://127.29.233.131:46211/ using document root <none> and password file <none>
I20260810 19:32:21.730386 2608 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:21.730530 2608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:21.730882 2608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:21.734058 2608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "7bbb0edb3e7c48be9e9f1bc4f204277c"
format_stamp: "Formatted at 2026-08-10 19:32:21 on dist-test-slave-tm1g"
server_key: "8189035cebb05f80b461c77cc5542c7e"
server_key_iv: "95cc37cf81021c2c272df61fac9ea885"
server_key_version: "encryptionkey@0"
I20260810 19:32:21.735018 2608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "7bbb0edb3e7c48be9e9f1bc4f204277c"
format_stamp: "Formatted at 2026-08-10 19:32:21 on dist-test-slave-tm1g"
server_key: "8189035cebb05f80b461c77cc5542c7e"
server_key_iv: "95cc37cf81021c2c272df61fac9ea885"
server_key_version: "encryptionkey@0"
I20260810 19:32:21.740727 2608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 19:32:21.745860 2624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:21.747362 2608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:32:21.747501 2608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "7bbb0edb3e7c48be9e9f1bc4f204277c"
format_stamp: "Formatted at 2026-08-10 19:32:21 on dist-test-slave-tm1g"
server_key: "8189035cebb05f80b461c77cc5542c7e"
server_key_iv: "95cc37cf81021c2c272df61fac9ea885"
server_key_version: "encryptionkey@0"
I20260810 19:32:21.747671 2608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:21.760550 2608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:21.764528 2608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:21.764781 2608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:21.765687 2608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:21.767048 2608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:21.767120 2608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:21.767204 2608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:21.767252 2608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:21.802740 2608 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:44999
I20260810 19:32:21.802783 2741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:44999 every 8 connection(s)
I20260810 19:32:21.804064 2608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:21.804495 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 2608
I20260810 19:32:21.804706 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:21.805358 30630 external_mini_cluster.cc:1428] Setting key aba32976c19a75aa9e4bed56ef7e0654
I20260810 19:32:21.811151 2455 heartbeater.cc:503] Master 127.29.233.190:42121 was elected leader, sending a full tablet report...
Aug 10 19:32:21 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:21.825935 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:21.807037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:39195 (local address 127.29.233.190:42121)
0810 19:32:21.807274 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:21.807279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:21.808818 (+ 1539us) server_negotiation.cc:407] Connection header received
0810 19:32:21.810844 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:21.810849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:21.810965 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:21.811052 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:21.813054 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.813846 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.815493 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.815897 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.816006 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:21.819485 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:21.819490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:21.819493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:21.819523 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:21.822922 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.823762 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.823765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.823768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.823894 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.824334 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.824337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.824339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.824619 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:21.824730 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:21.825519 (+ 789us) server_negotiation.cc:299] Negotiation successful
0810 19:32:21.825690 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 19:32:21.827149 2744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:21.807759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42121 (local address 127.29.233.131:39195)
0810 19:32:21.808649 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:21.808687 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:21.810556 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:21.811392 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:21.811405 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:21.812029 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:21.812836 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:21.812857 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.814006 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.814010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:21.815340 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:21.815353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.816094 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.816098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:21.816295 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:21.816953 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:21.816976 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:21.819251 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:21.823095 (+ 3844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:21.823103 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:21.823118 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:21.823636 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:21.824003 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:21.824007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:21.824009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:21.824228 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:21.824736 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:21.824749 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:21.825109 (+ 360us) client_negotiation.cc:769] Sending connection context
0810 19:32:21.825387 (+ 278us) client_negotiation.cc:240] Negotiation successful
0810 19:32:21.825843 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":369,"threads_started":1}
I20260810 19:32:21.828944 2742 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42121
I20260810 19:32:21.829284 2742 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:21.830032 2742 heartbeater.cc:511] Master 127.29.233.190:42121 requested a full tablet report, sending...
I20260810 19:32:21.831489 2251 ts_manager.cc:194] Registered new tserver with Master: 7bbb0edb3e7c48be9e9f1bc4f204277c (127.29.233.131:44999)
I20260810 19:32:21.832357 2251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:39195
I20260810 19:32:21.834612 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 19:32:21 dist-test-slave-tm1g krb5kdc[2201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 19:32:21 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:21.892127 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:21.876459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.29.233.129:46759)
0810 19:32:21.877094 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:21.877101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:21.877127 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:32:21.877188 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:21.877194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:21.877420 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:21.877553 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:21.879091 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.880177 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.881478 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.882037 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.882136 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:21.886217 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:21.886229 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:21.886244 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:21.886282 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:21.889224 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.889926 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.889933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.889936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.890067 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.890531 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.890535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.890537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.890869 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:21.891196 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:21.891493 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 19:32:21.891765 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":314,"threads_started":1}
Aug 10 19:32:21 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:21.910630 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:21.896752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51858 (local address 127.29.233.190:42121)
0810 19:32:21.897011 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:21.897016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:21.897113 (+ 97us) server_negotiation.cc:407] Connection header received
0810 19:32:21.897237 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:21.897240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:21.897303 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:21.897381 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:21.898717 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.899445 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.900714 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.901112 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.901203 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:21.905157 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:21.905162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:21.905165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:21.905199 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:21.907949 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.908602 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.908609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.908612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.908750 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.909122 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.909127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.909131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.909479 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:21.909748 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:21.910050 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 19:32:21.910271 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260810 19:32:21.911231 2250 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51858
I20260810 19:32:21.912246 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2316
I20260810 19:32:21.924695 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:21.913057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.29.233.129:46759)
0810 19:32:21.913220 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:21.913225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:21.914074 (+ 849us) server_negotiation.cc:407] Connection header received
0810 19:32:21.914191 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:21.914195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:21.914287 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:21.914420 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:21.915558 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.916234 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.917401 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:21.917807 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:21.917901 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:21.920063 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:21.920068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:21.920072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:21.920102 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:21.922618 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.923211 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.923215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.923217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.923329 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:21.923656 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:21.923659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:21.923661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:21.923895 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:21.924066 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:21.924304 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 19:32:21.924468 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 19:32:21.925279 2450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:22.010154 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 2316
I20260810 19:32:22.029445 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2463
Aug 10 19:32:22 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:32:22.047056 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:22.030592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.29.233.130:37845)
0810 19:32:22.031212 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:22.031219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:22.031247 (+ 28us) server_negotiation.cc:407] Connection header received
0810 19:32:22.031311 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:22.031318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:22.031546 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:22.031681 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:22.033083 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:22.034491 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:22.035895 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:22.036494 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:22.036619 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:22.041153 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:22.041165 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:22.041180 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:22.041218 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:22.044247 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:22.044952 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:22.044959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:22.044962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:22.045100 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:22.045498 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:22.045501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:22.045503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:22.045854 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:22.046118 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:22.046447 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 19:32:22.046650 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":316,"threads_started":1}
I20260810 19:32:22.048401 2597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:22.132577 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 2463
I20260810 19:32:22.152458 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2608
Aug 10 19:32:22 dist-test-slave-tm1g krb5kdc[2201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:32:22.169802 2759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:22.153435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.29.233.131:44999)
0810 19:32:22.154138 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:22.154145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:22.154175 (+ 30us) server_negotiation.cc:407] Connection header received
0810 19:32:22.154235 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:22.154242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:22.154544 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:22.154721 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:22.156041 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:22.157142 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:22.158528 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:22.159139 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:22.159295 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:22.163977 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:22.163990 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:22.164006 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:22.164039 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:22.167192 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:22.167876 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:22.167884 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:22.167886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:22.168018 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:22.168359 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:22.168362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:22.168364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:22.168694 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:22.168959 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:22.169177 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 19:32:22.169379 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":405,"threads_started":1}
I20260810 19:32:22.171262 2737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:22.255506 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 2608
I20260810 19:32:22.275215 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2215
I20260810 19:32:22.276523 2282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:22.384241 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 2215
2026-08-10T19:32:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4131 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:22 dist-test-slave-tm1g krb5kdc[2764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:22 dist-test-slave-tm1g krb5kdc[2764](info): set up 2 sockets
Aug 10 19:32:22 dist-test-slave-tm1g krb5kdc[2764](info): commencing operation
krb5kdc: starting...
W20260810 19:32:23.550976 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:23 dist-test-slave-tm1g krb5kdc[2764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:23.815724 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39679
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:39679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:24.154634 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:24.154927 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:24.155092 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:24.163117 2779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:24.163244 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:24.163331 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:24.163419 2779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:24.163511 2779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:24.173592 2779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:39679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:24.174983 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:24.176328 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:24.188179 2784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:24.188113 2787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:24.188151 2785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:24.188923 2779 server_base.cc:1034] running on GCE node
I20260810 19:32:24.189669 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:24.191090 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:24.192308 2779 hybrid_clock.cc:648] HybridClock initialized: now 1786390344192245 us; error 52 us; skew 500 ppm
I20260810 19:32:24.192687 2779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:24 dist-test-slave-tm1g krb5kdc[2764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390344, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:24.195937 2779 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:24.197139 2779 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:24.197280 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:24.197654 2779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:24.200767 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "b2014fc79878484c904ede5d90d4e319"
format_stamp: "Formatted at 2026-08-10 19:32:24 on dist-test-slave-tm1g"
server_key: "96ba10dc8d45105393a757f8d677a749"
server_key_iv: "3dbd8a98e9c0aab4f30f98f0375a0ad7"
server_key_version: "encryptionkey@0"
I20260810 19:32:24.201607 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "b2014fc79878484c904ede5d90d4e319"
format_stamp: "Formatted at 2026-08-10 19:32:24 on dist-test-slave-tm1g"
server_key: "96ba10dc8d45105393a757f8d677a749"
server_key_iv: "3dbd8a98e9c0aab4f30f98f0375a0ad7"
server_key_version: "encryptionkey@0"
I20260810 19:32:24.206894 2779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 19:32:24.210465 2793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:24.211776 2779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:24.211973 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "b2014fc79878484c904ede5d90d4e319"
format_stamp: "Formatted at 2026-08-10 19:32:24 on dist-test-slave-tm1g"
server_key: "96ba10dc8d45105393a757f8d677a749"
server_key_iv: "3dbd8a98e9c0aab4f30f98f0375a0ad7"
server_key_version: "encryptionkey@0"
I20260810 19:32:24.212201 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:24.239791 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:24.244108 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:24.244418 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:24.263216 2779 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:39679
I20260810 19:32:24.263239 2847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:39679 every 8 connection(s)
I20260810 19:32:24.264184 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:24.267999 2848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:24.271242 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 2779
I20260810 19:32:24.271518 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:24.272188 30630 external_mini_cluster.cc:1428] Setting key bc903af6a76f3a79b98d7dd2fc5d8d63
I20260810 19:32:24.281530 2848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319: Bootstrap starting.
I20260810 19:32:24.285526 2848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:24.286880 2848 log.cc:824] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319: Log is configured to *not* fsync() on all Append() calls
Aug 10 19:32:24 dist-test-slave-tm1g krb5kdc[2764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:24.290638 2848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319: No bootstrap required, opened a new log
I20260810 19:32:24.298900 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:24.274351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.29.233.190:39679)
0810 19:32:24.275275 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:24.275291 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:24.275341 (+ 50us) server_negotiation.cc:407] Connection header received
0810 19:32:24.277118 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:24.277149 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:24.277676 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:24.278188 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:24.279674 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:24.281626 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:24.283059 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:24.284156 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:24.284342 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:24.289648 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:24.289662 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:24.289684 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:24.289732 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:24.293805 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:24.294650 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:24.294660 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:24.294669 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:24.294882 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:24.295287 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:24.295293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:24.295296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:24.296045 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:24.296383 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:24.296924 (+ 541us) server_negotiation.cc:299] Negotiation successful
0810 19:32:24.297404 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":365,"threads_started":1}
I20260810 19:32:24.301447 2848 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2014fc79878484c904ede5d90d4e319" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39679 } }
I20260810 19:32:24.301894 2848 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:24.302032 2848 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2014fc79878484c904ede5d90d4e319, State: Initialized, Role: FOLLOWER
I20260810 19:32:24.302699 2848 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2014fc79878484c904ede5d90d4e319" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39679 } }
I20260810 19:32:24.302906 2848 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:24.303006 2848 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:24.303126 2848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:24.305233 2848 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2014fc79878484c904ede5d90d4e319" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39679 } }
I20260810 19:32:24.305708 2848 leader_election.cc:302] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2014fc79878484c904ede5d90d4e319; no voters:
I20260810 19:32:24.306231 2848 leader_election.cc:288] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:24.306411 2855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:24.307164 2855 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [term 1 LEADER]: Becoming Leader. State: Replica: b2014fc79878484c904ede5d90d4e319, State: Running, Role: LEADER
I20260810 19:32:24.307752 2855 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2014fc79878484c904ede5d90d4e319" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39679 } }
I20260810 19:32:24.308483 2848 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:24.310820 2856 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2014fc79878484c904ede5d90d4e319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2014fc79878484c904ede5d90d4e319" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39679 } } }
I20260810 19:32:24.311148 2856 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:24.314324 2857 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2014fc79878484c904ede5d90d4e319. Latest consensus state: current_term: 1 leader_uuid: "b2014fc79878484c904ede5d90d4e319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2014fc79878484c904ede5d90d4e319" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39679 } } }
I20260810 19:32:24.314808 2864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:24.314811 2857 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:24.319826 2864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:24.329150 2864 catalog_manager.cc:1347] Generated new cluster ID: 759064e2ec8e4ea28221f3d51bd82695
I20260810 19:32:24.329311 2864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:24.342582 2864 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:24.344143 2864 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:24.366575 2864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319: Generated new TSK 0
I20260810 19:32:24.367497 2864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:32:24.437914 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39679
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:24.800120 2879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:24.800424 2879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:24.800525 2879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:24.807986 2879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:24.808104 2879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:24.808219 2879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:24.818532 2879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:24.819948 2879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:24.821478 2879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:24.835067 2884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:24.835022 2887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:24.835098 2885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:24.836241 2879 server_base.cc:1034] running on GCE node
I20260810 19:32:24.836980 2879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:24.837977 2879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:24.839218 2879 hybrid_clock.cc:648] HybridClock initialized: now 1786390344839153 us; error 51 us; skew 500 ppm
I20260810 19:32:24.839579 2879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:24 dist-test-slave-tm1g krb5kdc[2764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390344, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:24.842825 2879 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:32:24.843961 2879 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:24.844050 2879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:24.844312 2879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:24.847434 2879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "319506b45e3a487aa2c6625d5d6a4679"
format_stamp: "Formatted at 2026-08-10 19:32:24 on dist-test-slave-tm1g"
server_key: "e82738d4a8e7ae20898ad010cd664dfc"
server_key_iv: "1efcb2103bb44c65cff59d728f735912"
server_key_version: "encryptionkey@0"
I20260810 19:32:24.848212 2879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "319506b45e3a487aa2c6625d5d6a4679"
format_stamp: "Formatted at 2026-08-10 19:32:24 on dist-test-slave-tm1g"
server_key: "e82738d4a8e7ae20898ad010cd664dfc"
server_key_iv: "1efcb2103bb44c65cff59d728f735912"
server_key_version: "encryptionkey@0"
I20260810 19:32:24.853397 2879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:32:24.856860 2894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:24.858217 2879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:24.858484 2879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "319506b45e3a487aa2c6625d5d6a4679"
format_stamp: "Formatted at 2026-08-10 19:32:24 on dist-test-slave-tm1g"
server_key: "e82738d4a8e7ae20898ad010cd664dfc"
server_key_iv: "1efcb2103bb44c65cff59d728f735912"
server_key_version: "encryptionkey@0"
I20260810 19:32:24.858680 2879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:24.890410 2879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:24.894452 2879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:24.894690 2879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:24.895628 2879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:24.897044 2879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:24.897135 2879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:24.897215 2879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:24.897275 2879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:24.932247 2879 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:43765
I20260810 19:32:24.932305 3018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:43765 every 8 connection(s)
I20260810 19:32:24.933166 2879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:24.937639 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 2879
I20260810 19:32:24.937885 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:24.938598 30630 external_mini_cluster.cc:1428] Setting key c20d12fe82cd840aa3a0fa3ae74c67d6
Aug 10 19:32:24 dist-test-slave-tm1g krb5kdc[2764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390344, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:24.953737 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:24.935764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:35647 (local address 127.29.233.190:39679)
0810 19:32:24.936290 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:24.936295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:24.937143 (+ 848us) server_negotiation.cc:407] Connection header received
0810 19:32:24.939192 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:24.939200 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:24.939304 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:24.939428 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:24.941227 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:24.941960 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:24.943616 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:24.944038 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:24.944178 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:24.947663 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:24.947668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:24.947671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:24.947703 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:24.950175 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:24.951175 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:24.951181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:24.951184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:24.951336 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:24.951850 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:24.951854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:24.951858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:24.952208 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:24.952352 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:24.953281 (+ 929us) server_negotiation.cc:299] Negotiation successful
0810 19:32:24.953468 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":291,"threads_started":1}
I20260810 19:32:24.955466 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:24.936227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39679 (local address 127.29.233.129:35647)
0810 19:32:24.936950 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:24.936992 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:24.938917 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:24.939673 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:24.939681 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:24.940407 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:24.941034 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:24.941057 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:24.942136 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:24.942140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:24.943458 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:24.943470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:24.944282 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:24.944289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:24.944523 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:24.945226 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:24.945251 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:24.947425 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:24.950349 (+ 2924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:24.950359 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:24.950375 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:24.951023 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:24.951466 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:24.951474 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:24.951481 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:24.951721 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:24.952342 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:24.952353 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:24.952832 (+ 479us) client_negotiation.cc:769] Sending connection context
0810 19:32:24.953162 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 19:32:24.953802 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":287,"threads_started":1}
I20260810 19:32:24.957175 3019 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39679
I20260810 19:32:24.957517 3019 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:24.958194 3019 heartbeater.cc:511] Master 127.29.233.190:39679 requested a full tablet report, sending...
I20260810 19:32:24.960366 2810 ts_manager.cc:194] Registered new tserver with Master: 319506b45e3a487aa2c6625d5d6a4679 (127.29.233.129:43765)
I20260810 19:32:24.962020 2810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:35647
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:32:24.994081 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39679
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:25.329989 3025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:25.330364 3025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:25.330541 3025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:25.337982 3025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:25.338145 3025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:25.338384 3025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:25.348438 3025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:25.349848 3025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:25.351231 3025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:25.364960 3031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:25.365023 3030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:25.365628 3025 server_base.cc:1034] running on GCE node
W20260810 19:32:25.365723 3033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:25.366542 3025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:25.367512 3025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:25.368707 3025 hybrid_clock.cc:648] HybridClock initialized: now 1786390345368672 us; error 33 us; skew 500 ppm
I20260810 19:32:25.369020 3025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:25 dist-test-slave-tm1g krb5kdc[2764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:25.380872 3025 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:32:25.382131 3025 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:25.382269 3025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:25.382652 3025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:25.385674 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "047fe0a688cc464d9b3dcc75640b04d2"
format_stamp: "Formatted at 2026-08-10 19:32:25 on dist-test-slave-tm1g"
server_key: "1b876898b0cbd2856dba5ec2fb2091fd"
server_key_iv: "734293b217e26bb4d924bd1916884d15"
server_key_version: "encryptionkey@0"
I20260810 19:32:25.386600 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "047fe0a688cc464d9b3dcc75640b04d2"
format_stamp: "Formatted at 2026-08-10 19:32:25 on dist-test-slave-tm1g"
server_key: "1b876898b0cbd2856dba5ec2fb2091fd"
server_key_iv: "734293b217e26bb4d924bd1916884d15"
server_key_version: "encryptionkey@0"
I20260810 19:32:25.391809 3025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 19:32:25.395413 3040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:25.396834 3025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:32:25.397019 3025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "047fe0a688cc464d9b3dcc75640b04d2"
format_stamp: "Formatted at 2026-08-10 19:32:25 on dist-test-slave-tm1g"
server_key: "1b876898b0cbd2856dba5ec2fb2091fd"
server_key_iv: "734293b217e26bb4d924bd1916884d15"
server_key_version: "encryptionkey@0"
I20260810 19:32:25.397264 3025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:25.433604 3025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:25.437945 3025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:25.438248 3025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:25.439299 3025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:25.440730 3025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:25.440857 3025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:25.441001 3025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:25.441095 3025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:25.476027 3025 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:42839
I20260810 19:32:25.476114 3160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:42839 every 8 connection(s)
I20260810 19:32:25.476938 3025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:25.478406 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 3025
I20260810 19:32:25.478624 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:25.479147 30630 external_mini_cluster.cc:1428] Setting key 31ad42b29ae1f8af479074e8d10abbd7
Aug 10 19:32:25 dist-test-slave-tm1g krb5kdc[2764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:25.497822 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:25.479585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:48273 (local address 127.29.233.190:39679)
0810 19:32:25.480105 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:25.480111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:25.481122 (+ 1011us) server_negotiation.cc:407] Connection header received
0810 19:32:25.483029 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:25.483039 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:25.483130 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:25.483252 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:25.484707 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:25.485897 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:25.487542 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:25.488291 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:25.488436 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:25.491998 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:25.492003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:25.492007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:25.492039 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:25.494745 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:25.495510 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:25.495514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:25.495517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:25.495641 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:25.496072 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:25.496084 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:25.496087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:25.496354 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:25.496543 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:25.497217 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 19:32:25.497443 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":280,"threads_started":1}
I20260810 19:32:25.498940 3165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:25.480113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39679 (local address 127.29.233.130:48273)
0810 19:32:25.480954 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:25.480999 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:25.482758 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:25.483436 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:25.483444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:25.483964 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:25.484520 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:25.484536 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:25.486053 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:25.486057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:25.487391 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:25.487409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:25.488501 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:25.488506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:25.488687 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:25.489393 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:25.489414 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:25.491770 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:25.494886 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:25.494896 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:25.494912 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:25.495367 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:25.495756 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:25.495761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:25.495765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:25.495943 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:25.496488 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:25.496497 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:25.496827 (+ 330us) client_negotiation.cc:769] Sending connection context
0810 19:32:25.497105 (+ 278us) client_negotiation.cc:240] Negotiation successful
0810 19:32:25.497622 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":379,"threads_started":1}
I20260810 19:32:25.500550 3161 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39679
I20260810 19:32:25.500882 3161 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:25.501546 3161 heartbeater.cc:511] Master 127.29.233.190:39679 requested a full tablet report, sending...
I20260810 19:32:25.503103 2810 ts_manager.cc:194] Registered new tserver with Master: 047fe0a688cc464d9b3dcc75640b04d2 (127.29.233.130:42839)
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
I20260810 19:32:25.503904 2810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:48273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:32:25.534086 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39679
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:25.870199 3168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:25.870651 3168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:25.870828 3168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:25.878219 3168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:25.878404 3168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:25.878614 3168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:25.888652 3168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:25.890100 3168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:25.891537 3168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:25.904939 3173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:25.905016 3176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:25.904933 3174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:25.905822 3168 server_base.cc:1034] running on GCE node
I20260810 19:32:25.906550 3168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:25.907509 3168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:25.908752 3168 hybrid_clock.cc:648] HybridClock initialized: now 1786390345908689 us; error 49 us; skew 500 ppm
I20260810 19:32:25.909117 3168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:25 dist-test-slave-tm1g krb5kdc[2764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:25.918943 3168 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:32:25.920239 3168 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:25.920377 3168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:25.920707 3168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:25.923772 3168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "c2d38c932fe24b1ab301c976264bb6fe"
format_stamp: "Formatted at 2026-08-10 19:32:25 on dist-test-slave-tm1g"
server_key: "e0b47acccb20d5b97414ebbe83ba2791"
server_key_iv: "acc936a2c8c5e7e5b59e72ec8c322cfb"
server_key_version: "encryptionkey@0"
I20260810 19:32:25.924610 3168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "c2d38c932fe24b1ab301c976264bb6fe"
format_stamp: "Formatted at 2026-08-10 19:32:25 on dist-test-slave-tm1g"
server_key: "e0b47acccb20d5b97414ebbe83ba2791"
server_key_iv: "acc936a2c8c5e7e5b59e72ec8c322cfb"
server_key_version: "encryptionkey@0"
I20260810 19:32:25.929817 3168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:32:25.933995 3182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:25.935746 3168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:25.935918 3168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "c2d38c932fe24b1ab301c976264bb6fe"
format_stamp: "Formatted at 2026-08-10 19:32:25 on dist-test-slave-tm1g"
server_key: "e0b47acccb20d5b97414ebbe83ba2791"
server_key_iv: "acc936a2c8c5e7e5b59e72ec8c322cfb"
server_key_version: "encryptionkey@0"
I20260810 19:32:25.936147 3168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:25.965132 3019 heartbeater.cc:503] Master 127.29.233.190:39679 was elected leader, sending a full tablet report...
I20260810 19:32:25.970957 3168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:25.975138 3168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:25.975466 3168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:25.976498 3168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:25.977869 3168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:25.977958 3168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:25.978044 3168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:25.978099 3168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:26.012535 3168 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:40105
I20260810 19:32:26.012566 3304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:40105 every 8 connection(s)
I20260810 19:32:26.013548 3168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:26.017740 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 3168
I20260810 19:32:26.017961 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:26.018478 30630 external_mini_cluster.cc:1428] Setting key ca9e50e6e10aff935e3ec194a9900dbb
Aug 10 19:32:26 dist-test-slave-tm1g krb5kdc[2764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:26.033968 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.016196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:52713 (local address 127.29.233.190:39679)
0810 19:32:26.016732 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.016737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.017612 (+ 875us) server_negotiation.cc:407] Connection header received
0810 19:32:26.019649 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.019658 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.019753 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.019871 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:26.021372 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.022174 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.023773 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.024212 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.024344 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.027949 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:26.027954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:26.027957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:26.027991 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:26.030752 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.031567 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.031572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.031582 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.031707 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.032145 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.032149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.032151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.032428 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:26.032596 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.033331 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.033562 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":259,"threads_started":1}
I20260810 19:32:26.035022 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.016725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39679 (local address 127.29.233.131:52713)
0810 19:32:26.017450 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:26.017494 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:26.019372 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:26.020108 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:26.020117 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:26.020633 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:26.021192 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.021209 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.022362 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.022367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.023615 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.023628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.024466 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.024474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.024686 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.025381 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:26.025403 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:26.027719 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:26.030940 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:26.030952 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:26.030975 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:26.031430 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:26.031820 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:26.031825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:26.031829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:26.032014 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:26.032560 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:26.032568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:26.032925 (+ 357us) client_negotiation.cc:769] Sending connection context
0810 19:32:26.033220 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 19:32:26.033772 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":299,"threads_started":1}
I20260810 19:32:26.036367 3305 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39679
I20260810 19:32:26.036656 3305 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:26.037271 3305 heartbeater.cc:511] Master 127.29.233.190:39679 requested a full tablet report, sending...
I20260810 19:32:26.038681 2810 ts_manager.cc:194] Registered new tserver with Master: c2d38c932fe24b1ab301c976264bb6fe (127.29.233.131:40105)
I20260810 19:32:26.039466 2810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:52713
I20260810 19:32:26.046802 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:32:26.069693 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.053885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38496 (local address 127.29.233.190:39679)
0810 19:32:26.054133 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.054142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.054407 (+ 265us) server_negotiation.cc:407] Connection header received
0810 19:32:26.054659 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.054668 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.054759 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.054883 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:26.056432 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.057743 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.059544 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.059958 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.060107 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.063335 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:26.063349 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:26.063352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:26.063387 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:26.066434 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.067122 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.067128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.067131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.067274 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.067667 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.067673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.067676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.068025 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:26.068318 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.069256 (+ 938us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.069458 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 19:32:26.075060 2810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:26.084523 2810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:26.120841 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.098283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.29.233.131:40105)
0810 19:32:26.112260 (+ 13977us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.112266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.112292 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:32:26.112371 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.112376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.112577 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.112790 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:26.113674 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.115402 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.118564 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.119548 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.119756 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.119961 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.120272 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.120435 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13793,"thread_start_us":13662,"threads_started":1}
I20260810 19:32:26.120885 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.098108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:40105 (local address 127.0.0.1:33736)
0810 19:32:26.111436 (+ 13328us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:26.111535 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:26.111704 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:26.112794 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:26.112802 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:26.112860 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:26.113375 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.113387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.115564 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.115571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.118401 (+ 2830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.118412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.119747 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.119754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.119963 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.120025 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 19:32:26.120209 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 19:32:26.120435 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13062,"mutex_wait_us":106,"thread_start_us":15029,"threads_started":1}
I20260810 19:32:26.123648 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.098706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:42839 (local address 127.0.0.1:37072)
0810 19:32:26.113492 (+ 14786us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:26.113519 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:26.113648 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:26.114363 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:26.114368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:26.114395 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:26.115053 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.115068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.117217 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.117225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.119522 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.119537 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.122712 (+ 3175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.122719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.122907 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.122951 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 19:32:26.123038 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 19:32:26.123179 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14622,"mutex_wait_us":2,"thread_start_us":12319,"threads_started":1}
I20260810 19:32:26.124176 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.114775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:43765 (local address 127.0.0.1:42066)
0810 19:32:26.116561 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:26.116578 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:26.116799 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:26.117411 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:26.117419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:26.117452 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:26.117983 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.117995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.119882 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.119888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.121555 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.121568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.123326 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.123333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.123509 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.123541 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 19:32:26.123639 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 19:32:26.123769 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1693,"mutex_wait_us":1,"thread_start_us":1522,"threads_started":1}
I20260810 19:32:26.124229 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.113964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.29.233.129:43765)
0810 19:32:26.114762 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.114770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.116704 (+ 1934us) server_negotiation.cc:407] Connection header received
0810 19:32:26.116971 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.116979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.117200 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.117407 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:26.118189 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.119752 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.121735 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.123098 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.123289 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.123467 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.123706 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.123890 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":484,"threads_started":1}
I20260810 19:32:26.124697 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.099579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37072 (local address 127.29.233.130:42839)
0810 19:32:26.109506 (+ 9927us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.109519 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.113641 (+ 4122us) server_negotiation.cc:407] Connection header received
0810 19:32:26.113832 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.113842 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.114073 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.115243 (+ 1170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:26.115489 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.117072 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.121126 (+ 4054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.122511 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.122645 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.122779 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.124136 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.124330 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9739,"thread_start_us":9627,"threads_started":1}
I20260810 19:32:26.126989 3235 tablet_service.cc:1467] Processing CreateTablet for tablet bdd819f0c66547119bb335d20120f20b (DEFAULT_TABLE table=test-table [id=c2f5c471dcf54aea898be36658cd0a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:26.127815 2946 tablet_service.cc:1467] Processing CreateTablet for tablet bdd819f0c66547119bb335d20120f20b (DEFAULT_TABLE table=test-table [id=c2f5c471dcf54aea898be36658cd0a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:26.128347 3235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd819f0c66547119bb335d20120f20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:26.128194 3092 tablet_service.cc:1467] Processing CreateTablet for tablet bdd819f0c66547119bb335d20120f20b (DEFAULT_TABLE table=test-table [id=c2f5c471dcf54aea898be36658cd0a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:26.129211 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd819f0c66547119bb335d20120f20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:26.129447 3092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd819f0c66547119bb335d20120f20b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:26.139969 3330 tablet_bootstrap.cc:492] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Bootstrap starting.
I20260810 19:32:26.143529 3330 tablet_bootstrap.cc:654] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:26.144567 3330 log.cc:824] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:26.147085 3332 tablet_bootstrap.cc:492] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Bootstrap starting.
I20260810 19:32:26.147984 3330 tablet_bootstrap.cc:492] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: No bootstrap required, opened a new log
I20260810 19:32:26.148265 3330 ts_tablet_manager.cc:1397] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 19:32:26.150534 3331 tablet_bootstrap.cc:492] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Bootstrap starting.
I20260810 19:32:26.150775 3332 tablet_bootstrap.cc:654] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:26.152035 3332 log.cc:824] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:26.153615 3331 tablet_bootstrap.cc:654] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:26.154637 3331 log.cc:824] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:26.155296 3332 tablet_bootstrap.cc:492] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: No bootstrap required, opened a new log
I20260810 19:32:26.155617 3332 ts_tablet_manager.cc:1397] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 19:32:26.157135 3331 tablet_bootstrap.cc:492] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: No bootstrap required, opened a new log
I20260810 19:32:26.157410 3331 ts_tablet_manager.cc:1397] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260810 19:32:26.158802 3330 raft_consensus.cc:348] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.159262 3330 raft_consensus.cc:374] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:26.159412 3330 raft_consensus.cc:729] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d38c932fe24b1ab301c976264bb6fe, State: Initialized, Role: FOLLOWER
I20260810 19:32:26.160290 3330 consensus_queue.cc:260] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.162748 3305 heartbeater.cc:503] Master 127.29.233.190:39679 was elected leader, sending a full tablet report...
I20260810 19:32:26.163419 3330 ts_tablet_manager.cc:1428] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260810 19:32:26.166890 3332 raft_consensus.cc:348] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.167276 3332 raft_consensus.cc:374] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:26.167405 3332 raft_consensus.cc:729] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319506b45e3a487aa2c6625d5d6a4679, State: Initialized, Role: FOLLOWER
I20260810 19:32:26.167364 3331 raft_consensus.cc:348] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.167801 3331 raft_consensus.cc:374] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:26.167965 3331 raft_consensus.cc:729] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047fe0a688cc464d9b3dcc75640b04d2, State: Initialized, Role: FOLLOWER
I20260810 19:32:26.168227 3332 consensus_queue.cc:260] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.168550 3331 consensus_queue.cc:260] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.170111 3332 ts_tablet_manager.cc:1428] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260810 19:32:26.171191 3161 heartbeater.cc:503] Master 127.29.233.190:39679 was elected leader, sending a full tablet report...
I20260810 19:32:26.171838 3331 ts_tablet_manager.cc:1428] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260810 19:32:26.185588 3020 tablet.cc:2369] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:32:26.229008 3163 tablet.cc:2369] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:32:26.265267 3306 tablet.cc:2369] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:32:26.299710 3338 raft_consensus.cc:482] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:26.300067 3338 raft_consensus.cc:504] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.302066 3338 leader_election.cc:288] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2d38c932fe24b1ab301c976264bb6fe (127.29.233.131:40105), 047fe0a688cc464d9b3dcc75640b04d2 (127.29.233.130:42839)
I20260810 19:32:26.309269 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.302602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:35905 (local address 127.29.233.131:40105)
0810 19:32:26.302823 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.302831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.303575 (+ 744us) server_negotiation.cc:407] Connection header received
0810 19:32:26.303789 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.303795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.303871 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.303949 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:26.304630 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.305426 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.307489 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.308272 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.308438 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.308527 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.308854 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.308996 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 19:32:26.309355 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.302672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:51331 (local address 127.29.233.130:42839)
0810 19:32:26.302874 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.302882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.303655 (+ 773us) server_negotiation.cc:407] Connection header received
0810 19:32:26.303857 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.303861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.303949 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.304049 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:26.304708 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.305429 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.307493 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.308304 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.308513 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.308586 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.308976 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.309110 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:32:26.309473 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.302563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:40105 (local address 127.29.233.129:35905)
0810 19:32:26.303458 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:26.303475 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:26.303627 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:26.304016 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:26.304024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:26.304051 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:26.304482 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.304493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.305556 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.305562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.307320 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.307332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.308501 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.308508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.308694 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.308718 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:32:26.308800 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 19:32:26.308920 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":57,"thread_start_us":447,"threads_started":1}
I20260810 19:32:26.309557 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.302572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:42839 (local address 127.29.233.129:51331)
0810 19:32:26.303509 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:26.303545 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:26.303710 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:26.304081 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:26.304086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:26.304111 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:26.304522 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.304532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.305592 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.305598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.307316 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:26.307329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.308628 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.308634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:26.308770 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.308830 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 19:32:26.308934 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 19:32:26.309051 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":77,"thread_start_us":476,"threads_started":1}
I20260810 19:32:26.310776 3256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd819f0c66547119bb335d20120f20b" candidate_uuid: "319506b45e3a487aa2c6625d5d6a4679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d38c932fe24b1ab301c976264bb6fe" is_pre_election: true
I20260810 19:32:26.310776 3112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd819f0c66547119bb335d20120f20b" candidate_uuid: "319506b45e3a487aa2c6625d5d6a4679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047fe0a688cc464d9b3dcc75640b04d2" is_pre_election: true
I20260810 19:32:26.311390 3256 raft_consensus.cc:2454] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 319506b45e3a487aa2c6625d5d6a4679 in term 0.
I20260810 19:32:26.311391 3112 raft_consensus.cc:2454] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 319506b45e3a487aa2c6625d5d6a4679 in term 0.
I20260810 19:32:26.312153 2897 leader_election.cc:302] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 047fe0a688cc464d9b3dcc75640b04d2, 319506b45e3a487aa2c6625d5d6a4679, c2d38c932fe24b1ab301c976264bb6fe; no voters:
I20260810 19:32:26.312616 3338 raft_consensus.cc:2781] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:26.312780 3338 raft_consensus.cc:482] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:26.312916 3338 raft_consensus.cc:3037] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:26.315263 3338 raft_consensus.cc:504] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.315914 3338 leader_election.cc:288] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [CANDIDATE]: Term 1 election: Requested vote from peers c2d38c932fe24b1ab301c976264bb6fe (127.29.233.131:40105), 047fe0a688cc464d9b3dcc75640b04d2 (127.29.233.130:42839)
I20260810 19:32:26.316434 3112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd819f0c66547119bb335d20120f20b" candidate_uuid: "319506b45e3a487aa2c6625d5d6a4679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047fe0a688cc464d9b3dcc75640b04d2"
I20260810 19:32:26.316498 3256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd819f0c66547119bb335d20120f20b" candidate_uuid: "319506b45e3a487aa2c6625d5d6a4679" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d38c932fe24b1ab301c976264bb6fe"
I20260810 19:32:26.316666 3112 raft_consensus.cc:3037] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:26.316711 3256 raft_consensus.cc:3037] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:26.319106 3256 raft_consensus.cc:2454] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 319506b45e3a487aa2c6625d5d6a4679 in term 1.
I20260810 19:32:26.319108 3112 raft_consensus.cc:2454] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 319506b45e3a487aa2c6625d5d6a4679 in term 1.
I20260810 19:32:26.319562 2898 leader_election.cc:302] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 319506b45e3a487aa2c6625d5d6a4679, c2d38c932fe24b1ab301c976264bb6fe; no voters:
I20260810 19:32:26.319892 3338 raft_consensus.cc:2781] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:26.320399 3338 raft_consensus.cc:686] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 1 LEADER]: Becoming Leader. State: Replica: 319506b45e3a487aa2c6625d5d6a4679, State: Running, Role: LEADER
I20260810 19:32:26.320935 3338 consensus_queue.cc:237] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } }
I20260810 19:32:26.325481 2810 catalog_manager.cc:5495] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 reported cstate change: term changed from 0 to 1, leader changed from <none> to 319506b45e3a487aa2c6625d5d6a4679 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "319506b45e3a487aa2c6625d5d6a4679" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "319506b45e3a487aa2c6625d5d6a4679" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:26.348335 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.342976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.29.233.129:43765)
0810 19:32:26.343164 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.343168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.343186 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:32:26.343286 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.343290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.343405 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.343540 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:26.344027 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.344783 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.346141 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.346657 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.346753 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.347252 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:26.347837 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:26.347964 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.348045 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.348116 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 19:32:26.357295 3256 raft_consensus.cc:1261] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 1 FOLLOWER]: Refusing update from remote peer 319506b45e3a487aa2c6625d5d6a4679: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:26.357295 3112 raft_consensus.cc:1261] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 1 FOLLOWER]: Refusing update from remote peer 319506b45e3a487aa2c6625d5d6a4679: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:26.358348 3342 consensus_queue.cc:1035] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047fe0a688cc464d9b3dcc75640b04d2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:26.358738 3338 consensus_queue.cc:1035] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d38c932fe24b1ab301c976264bb6fe" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:26.373449 3348 mvcc.cc:205] Tried to move back new op lower bound from 7317054858668011520 to 7317054858534092800. Current Snapshot: MvccSnapshot[applied={T|T < 7317054858668011520}]
I20260810 19:32:26.377620 3347 mvcc.cc:205] Tried to move back new op lower bound from 7317054858668011520 to 7317054858534092800. Current Snapshot: MvccSnapshot[applied={T|T < 7317054858668011520 or (T in {7317054858668011520})}]
I20260810 19:32:26.391396 2810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:26.391748 2810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:26.395753 2810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2014fc79878484c904ede5d90d4e319: Sending DeleteTablet for 3 replicas of tablet bdd819f0c66547119bb335d20120f20b
I20260810 19:32:26.397079 3235 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd819f0c66547119bb335d20120f20b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:26 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:33736
I20260810 19:32:26.397152 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd819f0c66547119bb335d20120f20b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:26 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:42066
I20260810 19:32:26.397298 3092 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd819f0c66547119bb335d20120f20b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:26 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:37072
I20260810 19:32:26.398195 3358 tablet_replica.cc:330] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: stopping tablet replica
I20260810 19:32:26.398712 3358 raft_consensus.cc:2229] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:26.399168 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2879
I20260810 19:32:26.399317 3358 raft_consensus.cc:2258] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:26.399916 3357 tablet_replica.cc:330] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: stopping tablet replica
I20260810 19:32:26.400326 3357 raft_consensus.cc:2229] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:26.398833 3359 tablet_replica.cc:330] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: stopping tablet replica
I20260810 19:32:26.401889 3359 raft_consensus.cc:2229] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:26.402561 3359 raft_consensus.cc:2258] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:26.403069 3357 raft_consensus.cc:2258] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:26.403394 3358 ts_tablet_manager.cc:1905] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:26.405927 3359 ts_tablet_manager.cc:1905] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:26.410832 3358 ts_tablet_manager.cc:1918] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:26.411123 3358 log.cc:1196] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/bdd819f0c66547119bb335d20120f20b
I20260810 19:32:26.411422 3357 ts_tablet_manager.cc:1905] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:26.411773 3358 ts_tablet_manager.cc:1939] T bdd819f0c66547119bb335d20120f20b P c2d38c932fe24b1ab301c976264bb6fe: Deleting consensus metadata
I20260810 19:32:26.414073 2797 catalog_manager.cc:4841] TS c2d38c932fe24b1ab301c976264bb6fe (127.29.233.131:40105): tablet bdd819f0c66547119bb335d20120f20b (table test-table [id=c2f5c471dcf54aea898be36658cd0a3a]) successfully deleted
I20260810 19:32:26.420114 3359 ts_tablet_manager.cc:1918] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:26.420410 3359 log.cc:1196] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/bdd819f0c66547119bb335d20120f20b
I20260810 19:32:26.420397 3357 ts_tablet_manager.cc:1918] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:26.421034 3357 log.cc:1196] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/bdd819f0c66547119bb335d20120f20b
I20260810 19:32:26.421109 3359 ts_tablet_manager.cc:1939] T bdd819f0c66547119bb335d20120f20b P 047fe0a688cc464d9b3dcc75640b04d2: Deleting consensus metadata
I20260810 19:32:26.421772 3357 ts_tablet_manager.cc:1939] T bdd819f0c66547119bb335d20120f20b P 319506b45e3a487aa2c6625d5d6a4679: Deleting consensus metadata
I20260810 19:32:26.423290 2796 catalog_manager.cc:4841] TS 047fe0a688cc464d9b3dcc75640b04d2 (127.29.233.130:42839): tablet bdd819f0c66547119bb335d20120f20b (table test-table [id=c2f5c471dcf54aea898be36658cd0a3a]) successfully deleted
I20260810 19:32:26.423748 2797 catalog_manager.cc:4841] TS 319506b45e3a487aa2c6625d5d6a4679 (127.29.233.129:43765): tablet bdd819f0c66547119bb335d20120f20b (table test-table [id=c2f5c471dcf54aea898be36658cd0a3a]) successfully deleted
Aug 10 19:32:26 dist-test-slave-tm1g krb5kdc[2764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:26.432456 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.400384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42090 (local address 127.29.233.129:43765)
0810 19:32:26.400586 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.400592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.413019 (+ 12427us) server_negotiation.cc:407] Connection header received
0810 19:32:26.413583 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.413589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.413677 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.413754 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:26.415691 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.416970 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.418962 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.419665 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.419758 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.425213 (+ 5455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:26.425235 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:26.425255 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:26.425336 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:26.429273 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.430003 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.430014 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.430018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.430190 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.430647 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.430653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.430657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.431152 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:26.431458 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.431739 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.432029 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:32:26.433243 3014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:26.545147 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 2879
I20260810 19:32:26.568264 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 3025
Aug 10 19:32:26 dist-test-slave-tm1g krb5kdc[2764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:32:26.583773 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.569187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.29.233.130:42839)
0810 19:32:26.569396 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.569400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.569420 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:26.569557 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.569561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.569626 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.569716 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:26.571202 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.571997 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.573351 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.573779 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.573892 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.578073 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:26.578090 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:26.578107 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:26.578146 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:26.581084 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.581793 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.581800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.581803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.581934 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.582259 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.582262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.582264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.582700 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:26.582933 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.583312 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.583500 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:32:26.584542 3155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:26.676539 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 3025
I20260810 19:32:26.696758 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 3168
Aug 10 19:32:26 dist-test-slave-tm1g krb5kdc[2764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:32:26.712977 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:26.697633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.29.233.131:40105)
0810 19:32:26.697868 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:26.697875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:26.697898 (+ 23us) server_negotiation.cc:407] Connection header received
0810 19:32:26.698103 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:26.698116 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:26.698182 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:26.698306 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:26.699650 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.700396 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.701670 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:26.702132 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:26.702274 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:26.707054 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:26.707072 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:26.707090 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:26.707126 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:26.710109 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.710919 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.710927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.710929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.711084 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:26.711525 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:26.711528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:26.711530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:26.711876 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:26.712123 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:26.712418 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 19:32:26.712599 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 19:32:26.713725 3300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:26.806690 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 3168
I20260810 19:32:26.826915 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 2779
I20260810 19:32:26.828137 2843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:26.939788 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 2779
2026-08-10T19:32:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4555 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T19:32:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:27Z Disabled control of system clock
I20260810 19:32:27.023795 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37141
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:37141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 19:32:27.369124 3369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:27.369439 3369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:27.369525 3369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:27.377115 3369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:27.377229 3369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:27.377269 3369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:27.377319 3369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:27.377368 3369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:27.387168 3369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:37141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:27.388522 3369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:27.390453 3369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:27.402716 3379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:27.402808 3375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:27.402784 3376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:27.403789 3369 server_base.cc:1034] running on GCE node
I20260810 19:32:27.404680 3369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:27.406201 3369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:27.407447 3369 hybrid_clock.cc:648] HybridClock initialized: now 1786390347407390 us; error 52 us; skew 500 ppm
I20260810 19:32:27.407857 3369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:27.410516 3369 webserver.cc:460] Webserver started at http://127.29.233.190:38411/ using document root <none> and password file <none>
I20260810 19:32:27.411281 3369 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:27.411371 3369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:27.411749 3369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:27.415259 3369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "0bd45c388962461593b87d375ae186a3"
format_stamp: "Formatted at 2026-08-10 19:32:27 on dist-test-slave-tm1g"
server_key: "057814c7a6e859821da6b36cafcb41ef"
server_key_iv: "d5e239d36f861447c2892bc064242c5b"
server_key_version: "encryptionkey@0"
I20260810 19:32:27.416100 3369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "0bd45c388962461593b87d375ae186a3"
format_stamp: "Formatted at 2026-08-10 19:32:27 on dist-test-slave-tm1g"
server_key: "057814c7a6e859821da6b36cafcb41ef"
server_key_iv: "d5e239d36f861447c2892bc064242c5b"
server_key_version: "encryptionkey@0"
I20260810 19:32:27.421552 3369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:32:27.425239 3385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:27.426718 3369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 19:32:27.426910 3369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "0bd45c388962461593b87d375ae186a3"
format_stamp: "Formatted at 2026-08-10 19:32:27 on dist-test-slave-tm1g"
server_key: "057814c7a6e859821da6b36cafcb41ef"
server_key_iv: "d5e239d36f861447c2892bc064242c5b"
server_key_version: "encryptionkey@0"
I20260810 19:32:27.427155 3369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:27.433391 3369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:27.434283 3369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:27.434746 3369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:27.455202 3369 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:37141
I20260810 19:32:27.455238 3443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:37141 every 8 connection(s)
I20260810 19:32:27.456694 3369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:27.457419 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 3369
I20260810 19:32:27.457669 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:27.458597 30630 external_mini_cluster.cc:1428] Setting key 2f523eed8cc273a8378c994685e16bc5
I20260810 19:32:27.461788 3444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:27.469405 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:27.460849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.29.233.190:37141)
0810 19:32:27.462089 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:27.462112 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:27.462169 (+ 57us) server_negotiation.cc:407] Connection header received
0810 19:32:27.464768 (+ 2599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:27.464797 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:27.465427 (+ 630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:27.465931 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:27.467100 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:27.467111 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:27.467133 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:27.467167 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:27.467283 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:27.467308 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:27.467564 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:27.467709 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 19:32:27.468008 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":450,"threads_started":1}
I20260810 19:32:27.476545 3444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3: Bootstrap starting.
I20260810 19:32:27.480165 3444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:27.481495 3444 log.cc:824] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:27.484515 3444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3: No bootstrap required, opened a new log
I20260810 19:32:27.492909 3444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd45c388962461593b87d375ae186a3" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37141 } }
I20260810 19:32:27.493309 3444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:27.493389 3444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd45c388962461593b87d375ae186a3, State: Initialized, Role: FOLLOWER
I20260810 19:32:27.493942 3444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd45c388962461593b87d375ae186a3" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37141 } }
I20260810 19:32:27.494163 3444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:27.494367 3444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:27.494539 3444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:27.497397 3444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd45c388962461593b87d375ae186a3" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37141 } }
I20260810 19:32:27.497917 3444 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd45c388962461593b87d375ae186a3; no voters:
I20260810 19:32:27.498594 3444 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:27.498754 3450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:27.499462 3450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [term 1 LEADER]: Becoming Leader. State: Replica: 0bd45c388962461593b87d375ae186a3, State: Running, Role: LEADER
I20260810 19:32:27.500054 3450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd45c388962461593b87d375ae186a3" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37141 } }
I20260810 19:32:27.500890 3444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:27.503118 3452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd45c388962461593b87d375ae186a3. Latest consensus state: current_term: 1 leader_uuid: "0bd45c388962461593b87d375ae186a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd45c388962461593b87d375ae186a3" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37141 } } }
I20260810 19:32:27.503546 3452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:27.503141 3451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bd45c388962461593b87d375ae186a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd45c388962461593b87d375ae186a3" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37141 } } }
I20260810 19:32:27.503780 3451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:27.507807 3457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:27.514813 3457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:27.524602 3457 catalog_manager.cc:1347] Generated new cluster ID: 017cf5912abe4c569271e5b1be0ef40a
I20260810 19:32:27.524746 3457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:27.545787 3457 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:27.546393 3457 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:27.561213 3457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3: Generated new TSK 0
I20260810 19:32:27.562448 3457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:32:27.568831 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37141
--builtin_ntp_servers=127.29.233.148:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 19:32:27.918958 3471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:27.919252 3471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:27.919355 3471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:27.926973 3471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:27.927085 3471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:27.927197 3471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:27.936864 3471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:27.938243 3471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:27.940264 3471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:27.954399 3476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:27.954488 3477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:27.954581 3479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:27.955688 3471 server_base.cc:1034] running on GCE node
I20260810 19:32:27.956431 3471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:27.957486 3471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:27.958727 3471 hybrid_clock.cc:648] HybridClock initialized: now 1786390347958666 us; error 50 us; skew 500 ppm
I20260810 19:32:27.959100 3471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:27.961555 3471 webserver.cc:460] Webserver started at http://127.29.233.129:38827/ using document root <none> and password file <none>
I20260810 19:32:27.962409 3471 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:27.962549 3471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:27.962913 3471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:27.965982 3471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "c87b92f304d84dc784ac9db34ee00f8f"
format_stamp: "Formatted at 2026-08-10 19:32:27 on dist-test-slave-tm1g"
server_key: "dc0a118901911afe837246e8234cf805"
server_key_iv: "71cea1b81e97c670f0d8d44db8c98118"
server_key_version: "encryptionkey@0"
I20260810 19:32:27.966845 3471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "c87b92f304d84dc784ac9db34ee00f8f"
format_stamp: "Formatted at 2026-08-10 19:32:27 on dist-test-slave-tm1g"
server_key: "dc0a118901911afe837246e8234cf805"
server_key_iv: "71cea1b81e97c670f0d8d44db8c98118"
server_key_version: "encryptionkey@0"
I20260810 19:32:27.972209 3471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 19:32:27.975759 3486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:27.977113 3471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:32:27.977312 3471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "c87b92f304d84dc784ac9db34ee00f8f"
format_stamp: "Formatted at 2026-08-10 19:32:27 on dist-test-slave-tm1g"
server_key: "dc0a118901911afe837246e8234cf805"
server_key_iv: "71cea1b81e97c670f0d8d44db8c98118"
server_key_version: "encryptionkey@0"
I20260810 19:32:27.977553 3471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:27.983745 3471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:27.984604 3471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:27.984901 3471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:27.985893 3471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:27.987403 3471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:27.987535 3471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:27.987665 3471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:27.987752 3471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:28.022746 3471 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:46337
I20260810 19:32:28.022840 3610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:46337 every 8 connection(s)
I20260810 19:32:28.024168 3471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:28.024755 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 3471
I20260810 19:32:28.024983 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:28.025949 30630 external_mini_cluster.cc:1428] Setting key f6203ba32bbb30d4a9586cc20966d22f
I20260810 19:32:28.029407 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37141
--builtin_ntp_servers=127.29.233.148:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 19:32:28.034569 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.027241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:56473 (local address 127.29.233.190:37141)
0810 19:32:28.027744 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:28.027750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:28.029250 (+ 1500us) server_negotiation.cc:407] Connection header received
0810 19:32:28.032089 (+ 2839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:28.032093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:28.032148 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:28.032299 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:28.033803 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:28.033808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:28.033811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:28.033827 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:28.033872 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:28.033884 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:28.033991 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:28.034210 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 19:32:28.034262 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":261,"threads_started":1}
I20260810 19:32:28.035912 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.027864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37141 (local address 127.29.233.129:56473)
0810 19:32:28.029072 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:28.029131 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:28.031787 (+ 2656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:28.032561 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:28.032576 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:28.032704 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:28.033438 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:28.033444 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:28.033498 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:28.033619 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:28.033985 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:28.033997 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:28.034008 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 19:32:28.034292 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 19:32:28.034614 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":382,"threads_started":1}
I20260810 19:32:28.037580 3611 heartbeater.cc:347] Connected to a master server at 127.29.233.190:37141
I20260810 19:32:28.037927 3611 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:28.038635 3611 heartbeater.cc:511] Master 127.29.233.190:37141 requested a full tablet report, sending...
I20260810 19:32:28.040993 3405 ts_manager.cc:194] Registered new tserver with Master: c87b92f304d84dc784ac9db34ee00f8f (127.29.233.129:46337)
W20260810 19:32:28.377321 3617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:28.377645 3617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:28.377799 3617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:28.385334 3617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:28.385449 3617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:28.385634 3617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:28.394994 3617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:28.396384 3617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:28.398257 3617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:28.412292 3626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:28.412441 3622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:28.412369 3623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:28.413139 3617 server_base.cc:1034] running on GCE node
I20260810 19:32:28.413858 3617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:28.414897 3617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:28.416159 3617 hybrid_clock.cc:648] HybridClock initialized: now 1786390348416112 us; error 62 us; skew 500 ppm
I20260810 19:32:28.416522 3617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:28.419096 3617 webserver.cc:460] Webserver started at http://127.29.233.130:37541/ using document root <none> and password file <none>
I20260810 19:32:28.419860 3617 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:28.420003 3617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:28.420359 3617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:28.423411 3617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "2cc058eede7d479daae9aec83b13f40f"
format_stamp: "Formatted at 2026-08-10 19:32:28 on dist-test-slave-tm1g"
server_key: "a5e594dd29d99eeb25cf3abd67f29c81"
server_key_iv: "589f2d8952865451559b20a04aa0f584"
server_key_version: "encryptionkey@0"
I20260810 19:32:28.424244 3617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "2cc058eede7d479daae9aec83b13f40f"
format_stamp: "Formatted at 2026-08-10 19:32:28 on dist-test-slave-tm1g"
server_key: "a5e594dd29d99eeb25cf3abd67f29c81"
server_key_iv: "589f2d8952865451559b20a04aa0f584"
server_key_version: "encryptionkey@0"
I20260810 19:32:28.429693 3617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:32:28.433697 3634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:28.435256 3617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:32:28.435456 3617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "2cc058eede7d479daae9aec83b13f40f"
format_stamp: "Formatted at 2026-08-10 19:32:28 on dist-test-slave-tm1g"
server_key: "a5e594dd29d99eeb25cf3abd67f29c81"
server_key_iv: "589f2d8952865451559b20a04aa0f584"
server_key_version: "encryptionkey@0"
I20260810 19:32:28.435708 3617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:28.441618 3617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:28.442582 3617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:28.442866 3617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:28.443823 3617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:28.445278 3617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:28.445433 3617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:28.445576 3617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:28.445670 3617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:28.480832 3617 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:35433
I20260810 19:32:28.480926 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:35433 every 8 connection(s)
I20260810 19:32:28.482311 3617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:28.489012 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 3617
I20260810 19:32:28.489392 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:28.490056 30630 external_mini_cluster.cc:1428] Setting key 8fcfbef703f3b4c10fe510974dd8b6ab
I20260810 19:32:28.491561 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.484973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:39341 (local address 127.29.233.190:37141)
0810 19:32:28.485172 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:28.485177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:28.486665 (+ 1488us) server_negotiation.cc:407] Connection header received
0810 19:32:28.488657 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:28.488663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:28.488751 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:28.488912 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:28.490738 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:28.490742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:28.490746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:28.490762 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:28.490797 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:28.490809 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:28.490962 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:28.491161 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 19:32:28.491236 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:32:28.492849 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.485506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37141 (local address 127.29.233.130:39341)
0810 19:32:28.486500 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:28.486546 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:28.488172 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:28.489277 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:28.489291 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:28.489479 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:28.490389 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:28.490392 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:28.490433 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:28.490530 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:28.490923 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:28.490935 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:28.490943 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 19:32:28.491318 (+ 375us) client_negotiation.cc:240] Negotiation successful
0810 19:32:28.491613 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":282,"threads_started":1}
I20260810 19:32:28.493582 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37141
--builtin_ntp_servers=127.29.233.148:43877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 19:32:28.494347 3763 heartbeater.cc:347] Connected to a master server at 127.29.233.190:37141
I20260810 19:32:28.494685 3763 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:28.495285 3763 heartbeater.cc:511] Master 127.29.233.190:37141 requested a full tablet report, sending...
I20260810 19:32:28.497004 3405 ts_manager.cc:194] Registered new tserver with Master: 2cc058eede7d479daae9aec83b13f40f (127.29.233.130:35433)
W20260810 19:32:28.839375 3766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:28.839726 3766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:28.839910 3766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:28.847360 3766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:28.847476 3766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:28.847591 3766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:28.856848 3766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:28.858470 3766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:28.860410 3766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:28.874732 3776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:28.874709 3773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:28.874720 3771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:28.875444 3766 server_base.cc:1034] running on GCE node
I20260810 19:32:28.876127 3766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:28.877102 3766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:28.878420 3766 hybrid_clock.cc:648] HybridClock initialized: now 1786390348878375 us; error 30 us; skew 500 ppm
I20260810 19:32:28.878803 3766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:28.881256 3766 webserver.cc:460] Webserver started at http://127.29.233.131:40009/ using document root <none> and password file <none>
I20260810 19:32:28.882073 3766 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:28.882191 3766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:28.882627 3766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:28.885676 3766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "695b0c993c9e4ff7b985d9e6bd64ffed"
format_stamp: "Formatted at 2026-08-10 19:32:28 on dist-test-slave-tm1g"
server_key: "c3461a345209d044bb020d3b3dca1ec3"
server_key_iv: "515ffd351be0c9b904448f462ddd8128"
server_key_version: "encryptionkey@0"
I20260810 19:32:28.886595 3766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "695b0c993c9e4ff7b985d9e6bd64ffed"
format_stamp: "Formatted at 2026-08-10 19:32:28 on dist-test-slave-tm1g"
server_key: "c3461a345209d044bb020d3b3dca1ec3"
server_key_iv: "515ffd351be0c9b904448f462ddd8128"
server_key_version: "encryptionkey@0"
I20260810 19:32:28.892066 3766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:32:28.895715 3783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:28.897110 3766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:28.897330 3766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "695b0c993c9e4ff7b985d9e6bd64ffed"
format_stamp: "Formatted at 2026-08-10 19:32:28 on dist-test-slave-tm1g"
server_key: "c3461a345209d044bb020d3b3dca1ec3"
server_key_iv: "515ffd351be0c9b904448f462ddd8128"
server_key_version: "encryptionkey@0"
I20260810 19:32:28.897576 3766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:28.903499 3766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:28.904304 3766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:28.904594 3766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:28.905556 3766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:28.907064 3766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:28.907187 3766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:28.907327 3766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:28.907423 3766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:28.942945 3766 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:45021
I20260810 19:32:28.942996 3900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:45021 every 8 connection(s)
I20260810 19:32:28.944303 3766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:28.949779 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 3766
I20260810 19:32:28.949946 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:28.950547 30630 external_mini_cluster.cc:1428] Setting key e96c301e7823fa6e9128271117e034e9
I20260810 19:32:28.952953 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.947027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:45201 (local address 127.29.233.190:37141)
0810 19:32:28.947221 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:28.947226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:28.948660 (+ 1434us) server_negotiation.cc:407] Connection header received
0810 19:32:28.950624 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:28.950629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:28.950713 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:28.950848 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:28.952150 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:28.952156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:28.952160 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:28.952181 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:28.952224 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:28.952242 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:28.952365 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:28.952559 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 19:32:28.952633 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 19:32:28.953961 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.947544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37141 (local address 127.29.233.131:45201)
0810 19:32:28.948490 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:28.948538 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:28.950262 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:28.951069 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:28.951077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:28.951169 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:28.951872 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:28.951874 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:28.951912 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:28.952002 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:28.952342 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:28.952349 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:28.952355 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 19:32:28.952628 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 19:32:28.952878 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":288,"threads_started":1}
I20260810 19:32:28.955292 3901 heartbeater.cc:347] Connected to a master server at 127.29.233.190:37141
I20260810 19:32:28.955547 3901 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:28.956059 3901 heartbeater.cc:511] Master 127.29.233.190:37141 requested a full tablet report, sending...
I20260810 19:32:28.957311 3405 ts_manager.cc:194] Registered new tserver with Master: 695b0c993c9e4ff7b985d9e6bd64ffed (127.29.233.131:45021)
I20260810 19:32:28.966144 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:32:28.974119 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.971386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.29.233.190:37141)
0810 19:32:28.971629 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:28.971638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:28.972151 (+ 513us) server_negotiation.cc:407] Connection header received
0810 19:32:28.972351 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:28.972363 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:28.972417 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:28.972531 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:28.973475 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:28.973480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:28.973483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:28.973499 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:28.973530 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:28.973542 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:28.973671 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:28.973787 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 19:32:28.973854 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260810 19:32:28.978514 3405 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:28.985646 3405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:29.004703 3918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.999225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37628 (local address 127.29.233.130:35433)
0810 19:32:29.001285 (+ 2060us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.001294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.001329 (+ 35us) server_negotiation.cc:407] Connection header received
0810 19:32:29.001462 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.001471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.001727 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.002672 (+ 945us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.003754 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.003762 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.003777 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.003808 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.003867 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.003887 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.004024 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.004243 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.004363 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1860,"thread_start_us":1739,"threads_started":1}
I20260810 19:32:29.004850 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.998647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:35433 (local address 127.0.0.1:37628)
0810 19:32:28.999856 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:28.999876 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:28.999979 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:29.001988 (+ 2009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:29.001997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:29.002097 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:29.003035 (+ 938us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:29.003041 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:29.003072 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:29.003167 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:29.004015 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:29.004027 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:29.004035 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 19:32:29.004231 (+ 196us) client_negotiation.cc:240] Negotiation successful
0810 19:32:29.004325 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":1440,"threads_started":1}
I20260810 19:32:29.005975 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.999338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45021 (local address 127.0.0.1:41644)
0810 19:32:29.001561 (+ 2223us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.001579 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:29.001852 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:29.002739 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:29.002744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:29.002766 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:29.004182 (+ 1416us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:29.004185 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:29.004201 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:29.004249 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:29.005352 (+ 1103us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:29.005357 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:29.005362 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 19:32:29.005500 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 19:32:29.005555 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2102,"thread_start_us":1919,"threads_started":1}
I20260810 19:32:29.006866 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.999650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.29.233.131:45021)
0810 19:32:29.000769 (+ 1119us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.000782 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.001777 (+ 995us) server_negotiation.cc:407] Connection header received
0810 19:32:29.002034 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.002042 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.002225 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.002395 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.004432 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.004440 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.004452 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.004470 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.005066 (+ 596us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.005099 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.005277 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.006344 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.006445 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":741,"threads_started":1}
I20260810 19:32:29.007331 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.997968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48450)
0810 19:32:28.998828 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:28.998874 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:28.999008 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:29.004407 (+ 5399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:29.004412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:29.004435 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:29.005320 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:29.005324 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:29.005340 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:29.005378 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:29.006718 (+ 1340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:29.006722 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:29.006728 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 19:32:29.006884 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 19:32:29.006936 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":338,"threads_started":1}
I20260810 19:32:29.007339 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:28.998140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.29.233.129:46337)
0810 19:32:29.003898 (+ 5758us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.003904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.003925 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:32:29.004019 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.004026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.004231 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.004920 (+ 689us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.006477 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.006484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.006495 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.006519 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.006569 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.006586 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.006729 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.006892 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.006988 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5629,"thread_start_us":5572,"threads_started":1}
I20260810 19:32:29.009498 3686 tablet_service.cc:1467] Processing CreateTablet for tablet 5628a00fced44b73b02ec60cd3efaf03 (DEFAULT_TABLE table=test-table [id=f2a3ddf4f46c40e595554a005335db6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:29.010180 3833 tablet_service.cc:1467] Processing CreateTablet for tablet 5628a00fced44b73b02ec60cd3efaf03 (DEFAULT_TABLE table=test-table [id=f2a3ddf4f46c40e595554a005335db6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:29.011127 3686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5628a00fced44b73b02ec60cd3efaf03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:29.011435 3833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5628a00fced44b73b02ec60cd3efaf03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:29.010921 3538 tablet_service.cc:1467] Processing CreateTablet for tablet 5628a00fced44b73b02ec60cd3efaf03 (DEFAULT_TABLE table=test-table [id=f2a3ddf4f46c40e595554a005335db6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:29.012193 3538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5628a00fced44b73b02ec60cd3efaf03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:29.028752 3921 tablet_bootstrap.cc:492] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Bootstrap starting.
I20260810 19:32:29.030202 3924 tablet_bootstrap.cc:492] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f: Bootstrap starting.
I20260810 19:32:29.035051 3923 tablet_bootstrap.cc:492] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Bootstrap starting.
I20260810 19:32:29.035965 3921 tablet_bootstrap.cc:654] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:29.037087 3924 tablet_bootstrap.cc:654] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:29.037273 3921 log.cc:824] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:29.038551 3924 log.cc:824] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:29.039564 3923 tablet_bootstrap.cc:654] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:29.041020 3923 log.cc:824] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:29.042470 3921 tablet_bootstrap.cc:492] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: No bootstrap required, opened a new log
I20260810 19:32:29.042476 3924 tablet_bootstrap.cc:492] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f: No bootstrap required, opened a new log
I20260810 19:32:29.042760 3921 ts_tablet_manager.cc:1397] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260810 19:32:29.042760 3924 ts_tablet_manager.cc:1397] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260810 19:32:29.043669 3923 tablet_bootstrap.cc:492] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: No bootstrap required, opened a new log
I20260810 19:32:29.043934 3923 ts_tablet_manager.cc:1397] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 19:32:29.044451 3611 heartbeater.cc:503] Master 127.29.233.190:37141 was elected leader, sending a full tablet report...
I20260810 19:32:29.052884 3923 raft_consensus.cc:348] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.053297 3923 raft_consensus.cc:374] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:29.053447 3923 raft_consensus.cc:729] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc058eede7d479daae9aec83b13f40f, State: Initialized, Role: FOLLOWER
I20260810 19:32:29.054106 3923 consensus_queue.cc:260] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.054385 3921 raft_consensus.cc:348] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.054522 3924 raft_consensus.cc:348] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.054947 3924 raft_consensus.cc:374] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:29.054948 3921 raft_consensus.cc:374] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:29.055083 3924 raft_consensus.cc:729] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87b92f304d84dc784ac9db34ee00f8f, State: Initialized, Role: FOLLOWER
I20260810 19:32:29.055083 3921 raft_consensus.cc:729] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 695b0c993c9e4ff7b985d9e6bd64ffed, State: Initialized, Role: FOLLOWER
I20260810 19:32:29.055286 3763 heartbeater.cc:503] Master 127.29.233.190:37141 was elected leader, sending a full tablet report...
I20260810 19:32:29.055685 3924 consensus_queue.cc:260] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.055685 3921 consensus_queue.cc:260] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.057693 3921 ts_tablet_manager.cc:1428] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260810 19:32:29.058279 3924 ts_tablet_manager.cc:1428] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260810 19:32:29.058792 3923 ts_tablet_manager.cc:1428] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Time spent starting tablet: real 0.015s user 0.005s sys 0.008s
I20260810 19:32:29.058950 3901 heartbeater.cc:503] Master 127.29.233.190:37141 was elected leader, sending a full tablet report...
I20260810 19:32:29.171556 3929 raft_consensus.cc:482] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:29.171968 3929 raft_consensus.cc:504] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.173831 3929 leader_election.cc:288] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 695b0c993c9e4ff7b985d9e6bd64ffed (127.29.233.131:45021), 2cc058eede7d479daae9aec83b13f40f (127.29.233.130:35433)
I20260810 19:32:29.177474 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.174243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:47335 (local address 127.29.233.131:45021)
0810 19:32:29.174479 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.174485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.175484 (+ 999us) server_negotiation.cc:407] Connection header received
0810 19:32:29.175607 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.175613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.175688 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.175790 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.176845 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.176851 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.176855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.176876 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.176915 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.176932 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.177079 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.177128 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.177177 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 19:32:29.177721 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.174292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45021 (local address 127.29.233.129:47335)
0810 19:32:29.175354 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.175371 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:29.175491 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:29.175836 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:29.175848 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:29.175875 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:29.176639 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:29.176642 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:29.176657 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:29.176695 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:29.177040 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:29.177045 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:29.177050 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 19:32:29.177178 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 19:32:29.177239 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":608,"threads_started":1}
I20260810 19:32:29.177794 3918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.174400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:34989 (local address 127.29.233.130:35433)
0810 19:32:29.174629 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.174637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.175633 (+ 996us) server_negotiation.cc:407] Connection header received
0810 19:32:29.175767 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.175775 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.175841 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.175953 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.177078 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.177086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.177093 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.177118 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.177161 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.177176 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.177298 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.177449 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.177509 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 19:32:29.178320 3934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.174391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:35433 (local address 127.29.233.129:34989)
0810 19:32:29.175466 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.175493 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:29.175626 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:29.175995 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:29.175999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:29.176020 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:29.176836 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:32:29.176839 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:29.176855 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:32:29.176897 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:29.177290 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:29.177299 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:29.177305 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 19:32:29.177455 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 19:32:29.177508 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":565,"threads_started":1}
I20260810 19:32:29.179024 3854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5628a00fced44b73b02ec60cd3efaf03" candidate_uuid: "c87b92f304d84dc784ac9db34ee00f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" is_pre_election: true
I20260810 19:32:29.179222 3710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5628a00fced44b73b02ec60cd3efaf03" candidate_uuid: "c87b92f304d84dc784ac9db34ee00f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc058eede7d479daae9aec83b13f40f" is_pre_election: true
I20260810 19:32:29.179566 3854 raft_consensus.cc:2454] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87b92f304d84dc784ac9db34ee00f8f in term 0.
I20260810 19:32:29.179790 3710 raft_consensus.cc:2454] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87b92f304d84dc784ac9db34ee00f8f in term 0.
I20260810 19:32:29.180264 3488 leader_election.cc:302] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695b0c993c9e4ff7b985d9e6bd64ffed, c87b92f304d84dc784ac9db34ee00f8f; no voters:
I20260810 19:32:29.180716 3929 raft_consensus.cc:2781] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:29.180881 3929 raft_consensus.cc:482] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:29.181003 3929 raft_consensus.cc:3037] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:29.183514 3929 raft_consensus.cc:504] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.184170 3929 leader_election.cc:288] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [CANDIDATE]: Term 1 election: Requested vote from peers 695b0c993c9e4ff7b985d9e6bd64ffed (127.29.233.131:45021), 2cc058eede7d479daae9aec83b13f40f (127.29.233.130:35433)
I20260810 19:32:29.184674 3710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5628a00fced44b73b02ec60cd3efaf03" candidate_uuid: "c87b92f304d84dc784ac9db34ee00f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc058eede7d479daae9aec83b13f40f"
I20260810 19:32:29.184715 3854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5628a00fced44b73b02ec60cd3efaf03" candidate_uuid: "c87b92f304d84dc784ac9db34ee00f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed"
I20260810 19:32:29.184895 3710 raft_consensus.cc:3037] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:29.184927 3854 raft_consensus.cc:3037] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:29.187260 3710 raft_consensus.cc:2454] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87b92f304d84dc784ac9db34ee00f8f in term 1.
I20260810 19:32:29.187335 3854 raft_consensus.cc:2454] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87b92f304d84dc784ac9db34ee00f8f in term 1.
I20260810 19:32:29.187779 3489 leader_election.cc:302] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc058eede7d479daae9aec83b13f40f, c87b92f304d84dc784ac9db34ee00f8f; no voters:
I20260810 19:32:29.188135 3929 raft_consensus.cc:2781] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:29.188597 3929 raft_consensus.cc:686] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [term 1 LEADER]: Becoming Leader. State: Replica: c87b92f304d84dc784ac9db34ee00f8f, State: Running, Role: LEADER
I20260810 19:32:29.189092 3929 consensus_queue.cc:237] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } }
I20260810 19:32:29.194077 3403 catalog_manager.cc:5495] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to c87b92f304d84dc784ac9db34ee00f8f (127.29.233.129). New cstate: current_term: 1 leader_uuid: "c87b92f304d84dc784ac9db34ee00f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87b92f304d84dc784ac9db34ee00f8f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 } health_report { overall_health: UNKNOWN } } }
W20260810 19:32:29.196178 3902 tablet.cc:2369] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:32:29.209228 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.207157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.29.233.129:46337)
0810 19:32:29.207326 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.207331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.207495 (+ 164us) server_negotiation.cc:407] Connection header received
0810 19:32:29.207667 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.207671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.207750 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.207835 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.208727 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.208731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.208735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.208749 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.208781 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.208793 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.208883 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.208981 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.209027 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 19:32:29.217734 3710 raft_consensus.cc:1261] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 1 FOLLOWER]: Refusing update from remote peer c87b92f304d84dc784ac9db34ee00f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:29.217733 3854 raft_consensus.cc:1261] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 1 FOLLOWER]: Refusing update from remote peer c87b92f304d84dc784ac9db34ee00f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:29.218845 3935 consensus_queue.cc:1035] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "695b0c993c9e4ff7b985d9e6bd64ffed" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:29.219280 3929 consensus_queue.cc:1035] T 5628a00fced44b73b02ec60cd3efaf03 P c87b92f304d84dc784ac9db34ee00f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc058eede7d479daae9aec83b13f40f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 35433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:29.232878 3939 mvcc.cc:205] Tried to move back new op lower bound from 7317054870384857088 to 7317054870282346496. Current Snapshot: MvccSnapshot[applied={T|T < 7317054870384857088}]
I20260810 19:32:29.233419 3938 mvcc.cc:205] Tried to move back new op lower bound from 7317054870384857088 to 7317054870282346496. Current Snapshot: MvccSnapshot[applied={T|T < 7317054870384857088}]
I20260810 19:32:29.245121 3403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:29.245447 3403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:29.249058 3403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bd45c388962461593b87d375ae186a3: Sending DeleteTablet for 3 replicas of tablet 5628a00fced44b73b02ec60cd3efaf03
I20260810 19:32:29.250417 3686 tablet_service.cc:1514] Processing DeleteTablet for tablet 5628a00fced44b73b02ec60cd3efaf03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:29 UTC) from {username='slave'} at 127.0.0.1:37628
I20260810 19:32:29.250468 3833 tablet_service.cc:1514] Processing DeleteTablet for tablet 5628a00fced44b73b02ec60cd3efaf03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:29 UTC) from {username='slave'} at 127.0.0.1:41644
I20260810 19:32:29.251464 3538 tablet_service.cc:1514] Processing DeleteTablet for tablet 5628a00fced44b73b02ec60cd3efaf03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:29 UTC) from {username='slave'} at 127.0.0.1:48450
I20260810 19:32:29.251554 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 3471
I20260810 19:32:29.269840 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.253196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.29.233.129:46337)
0810 19:32:29.253407 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.253420 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.266011 (+ 12591us) server_negotiation.cc:407] Connection header received
0810 19:32:29.266636 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.266645 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.266728 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.266819 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.268267 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.268276 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.268284 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.268312 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.268358 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.268377 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.268507 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.269313 (+ 806us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.269393 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 19:32:29.271421 3606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:29.271814 3948 tablet_replica.cc:330] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: stopping tablet replica
I20260810 19:32:29.272435 3948 raft_consensus.cc:2229] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:29.273131 3948 raft_consensus.cc:2258] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:29.274255 3949 tablet_replica.cc:330] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: stopping tablet replica
I20260810 19:32:29.274716 3949 raft_consensus.cc:2229] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:29.275398 3949 raft_consensus.cc:2258] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:29.276222 3948 ts_tablet_manager.cc:1905] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:29.279321 3949 ts_tablet_manager.cc:1905] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:29.283476 3948 ts_tablet_manager.cc:1918] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:29.283675 3948 log.cc:1196] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/5628a00fced44b73b02ec60cd3efaf03
I20260810 19:32:29.284236 3948 ts_tablet_manager.cc:1939] T 5628a00fced44b73b02ec60cd3efaf03 P 2cc058eede7d479daae9aec83b13f40f: Deleting consensus metadata
I20260810 19:32:29.286383 3388 catalog_manager.cc:4841] TS 2cc058eede7d479daae9aec83b13f40f (127.29.233.130:35433): tablet 5628a00fced44b73b02ec60cd3efaf03 (table test-table [id=f2a3ddf4f46c40e595554a005335db6b]) successfully deleted
I20260810 19:32:29.286832 3949 ts_tablet_manager.cc:1918] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:29.286998 3949 log.cc:1196] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/5628a00fced44b73b02ec60cd3efaf03
I20260810 19:32:29.287490 3949 ts_tablet_manager.cc:1939] T 5628a00fced44b73b02ec60cd3efaf03 P 695b0c993c9e4ff7b985d9e6bd64ffed: Deleting consensus metadata
I20260810 19:32:29.288900 3387 catalog_manager.cc:4841] TS 695b0c993c9e4ff7b985d9e6bd64ffed (127.29.233.131:45021): tablet 5628a00fced44b73b02ec60cd3efaf03 (table test-table [id=f2a3ddf4f46c40e595554a005335db6b]) successfully deleted
I20260810 19:32:29.373673 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 3471
I20260810 19:32:29.394637 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 3617
W20260810 19:32:29.394591 3387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.233.129:46337 (error 108)
I20260810 19:32:29.397253 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.396581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48480)
0810 19:32:29.396875 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.397061 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260810 19:32:29.397539 3387 catalog_manager.cc:4584] TS c87b92f304d84dc784ac9db34ee00f8f (127.29.233.129:46337): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5628a00fced44b73b02ec60cd3efaf03: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
I20260810 19:32:29.397991 3918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.395580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.29.233.130:35433)
0810 19:32:29.395784 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.395790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.395814 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:32:29.396008 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.396014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.396121 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.396213 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.397303 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.397307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.397311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.397326 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.397386 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.397402 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.397529 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.397692 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.397755 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 19:32:29.398777 3757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:29.451429 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.450969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48486)
0810 19:32:29.451167 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.451252 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 19:32:29.452533 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.452165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48502)
0810 19:32:29.452322 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.452417 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 19:32:29.492825 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 3617
I20260810 19:32:29.513032 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 3766
I20260810 19:32:29.513077 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.512549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48506)
0810 19:32:29.512820 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.512912 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260810 19:32:29.514168 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.513775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48520)
0810 19:32:29.513963 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.514049 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 19:32:29.516090 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.513873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41652 (local address 127.29.233.131:45021)
0810 19:32:29.514066 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:29.514072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:29.514092 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:29.514325 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:29.514331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:29.514466 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:29.514594 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:29.515440 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:29.515445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:29.515450 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:29.515476 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:29.515510 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:29.515521 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:29.515665 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:29.515748 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 19:32:29.515810 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 19:32:29.516733 3896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:29.595126 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.594513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48534)
0810 19:32:29.594784 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.594907 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 19:32:29.596308 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:29.595873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46337 (local address 127.0.0.1:48544)
0810 19:32:29.596091 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:29.596172 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:46337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260810 19:32:29.605728 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 3766
I20260810 19:32:29.625118 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 3369
I20260810 19:32:29.626536 3439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:29.734179 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 3369
2026-08-10T19:32:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2793 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 19:32:29.781128 30630 mini_oidc.cc:152] Starting JWKS server
I20260810 19:32:29.784087 30630 webserver.cc:460] Webserver started at https://127.0.0.1:33813/ using document root <none> and password file <none>
I20260810 19:32:29.784453 30630 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 19:32:29.785156 30630 webserver.cc:460] Webserver started at http://127.0.0.1:45755/ using document root <none> and password file <none>
2026-08-10T19:32:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:29Z Disabled control of system clock
I20260810 19:32:29.821579 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39745
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:39745
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33813/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/testchainca.cert with env {}
W20260810 19:32:30.149407 3962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:30.149771 3962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:30.149936 3962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:30.157100 3962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:30.157245 3962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:30.157341 3962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:30.157449 3962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:30.157536 3962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 19:32:30.157627 3962 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 19:32:30.157709 3962 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33813/jwks.json
I20260810 19:32:30.167922 3962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:39745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33813/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:30.169349 3962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:30.170724 3962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:30.182878 3967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:30.182971 3968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:30.182934 3971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:30.183828 3962 server_base.cc:1034] running on GCE node
I20260810 19:32:30.184563 3962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:30.185916 3962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:30.187155 3962 hybrid_clock.cc:648] HybridClock initialized: now 1786390350187094 us; error 44 us; skew 500 ppm
I20260810 19:32:30.187474 3962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:30.189798 3962 webserver.cc:460] Webserver started at http://127.29.233.190:43511/ using document root <none> and password file <none>
I20260810 19:32:30.190528 3962 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:30.190626 3962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:30.190879 3962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:30.193776 3962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "b4621d584241474183d34c7f8fcd7d63"
format_stamp: "Formatted at 2026-08-10 19:32:30 on dist-test-slave-tm1g"
server_key: "73af6cdd53888838b55ff4c8d2aef939"
server_key_iv: "3a5e0c59ca0af45d3217205fe2a7bfd0"
server_key_version: "encryptionkey@0"
I20260810 19:32:30.194612 3962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "b4621d584241474183d34c7f8fcd7d63"
format_stamp: "Formatted at 2026-08-10 19:32:30 on dist-test-slave-tm1g"
server_key: "73af6cdd53888838b55ff4c8d2aef939"
server_key_iv: "3a5e0c59ca0af45d3217205fe2a7bfd0"
server_key_version: "encryptionkey@0"
I20260810 19:32:30.199648 3962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 19:32:30.203140 3977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:30.204499 3962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:30.204639 3962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "b4621d584241474183d34c7f8fcd7d63"
format_stamp: "Formatted at 2026-08-10 19:32:30 on dist-test-slave-tm1g"
server_key: "73af6cdd53888838b55ff4c8d2aef939"
server_key_iv: "3a5e0c59ca0af45d3217205fe2a7bfd0"
server_key_version: "encryptionkey@0"
I20260810 19:32:30.204810 3962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:30.233939 3962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:30.234802 3962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:30.235098 3962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:30.253964 3962 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:39745
I20260810 19:32:30.253989 4031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:39745 every 8 connection(s)
I20260810 19:32:30.255350 3962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:30.259052 4032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:30.262796 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 3962
I20260810 19:32:30.263027 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:30.263871 30630 external_mini_cluster.cc:1428] Setting key 598546f779a2a2129f75dee2f884d313
I20260810 19:32:30.270715 4032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63: Bootstrap starting.
I20260810 19:32:30.274237 4032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:30.275470 4032 log.cc:824] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:30.278509 4032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63: No bootstrap required, opened a new log
I20260810 19:32:30.278909 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:30.266125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.29.233.190:39745)
0810 19:32:30.267475 (+ 1350us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:30.267489 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:30.267535 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:32:30.269304 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:30.269325 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:30.269737 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:30.270035 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:30.270695 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:30.272064 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:30.274023 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:30.274795 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:30.274927 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:30.276345 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:30.276363 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:30.276387 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:30.277078 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:30.277154 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:30.277176 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:30.277270 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:30.277550 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 19:32:30.277890 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":644,"threads_started":1}
I20260810 19:32:30.282586 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:32:30.287375 4032 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4621d584241474183d34c7f8fcd7d63" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39745 } }
I20260810 19:32:30.287690 4032 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:30.287766 4032 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4621d584241474183d34c7f8fcd7d63, State: Initialized, Role: FOLLOWER
I20260810 19:32:30.288236 4032 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4621d584241474183d34c7f8fcd7d63" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39745 } }
I20260810 19:32:30.288429 4032 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:30.288522 4032 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:30.288638 4032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:30.290799 4032 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4621d584241474183d34c7f8fcd7d63" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39745 } }
I20260810 19:32:30.291306 4032 leader_election.cc:302] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4621d584241474183d34c7f8fcd7d63; no voters:
I20260810 19:32:30.291855 4032 leader_election.cc:288] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:30.292044 4037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:30.293231 4037 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [term 1 LEADER]: Becoming Leader. State: Replica: b4621d584241474183d34c7f8fcd7d63, State: Running, Role: LEADER
I20260810 19:32:30.293747 4037 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4621d584241474183d34c7f8fcd7d63" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39745 } }
I20260810 19:32:30.294003 4032 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:30.297091 4038 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4621d584241474183d34c7f8fcd7d63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4621d584241474183d34c7f8fcd7d63" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39745 } } }
I20260810 19:32:30.297132 4039 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4621d584241474183d34c7f8fcd7d63. Latest consensus state: current_term: 1 leader_uuid: "b4621d584241474183d34c7f8fcd7d63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4621d584241474183d34c7f8fcd7d63" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39745 } } }
I20260810 19:32:30.297559 4038 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:30.297559 4039 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:30.298604 4047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:30.304378 4047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:30.313730 4047 catalog_manager.cc:1347] Generated new cluster ID: ccd6584ba6244606b110bed23960104f
I20260810 19:32:30.313844 4047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:30.352715 4047 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:30.354669 4047 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:30.373806 4047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4621d584241474183d34c7f8fcd7d63: Generated new TSK 0
I20260810 19:32:30.374925 4047 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 19:32:30.416996 4064 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 19:32:30.417256 4064 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 19:32:30.418223 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:30.396028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.29.233.190:39745)
0810 19:32:30.396285 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:30.396295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:30.396806 (+ 511us) server_negotiation.cc:407] Connection header received
0810 19:32:30.397214 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:30.397218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:30.397293 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:30.397438 (+ 145us) server_negotiation.cc:226] Negotiated authn=JWT
0810 19:32:30.398143 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:30.399015 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:30.401129 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:30.401634 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:30.401728 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:30.402462 (+ 734us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 19:32:30.417704 (+ 15242us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33813/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 19:32:30.417995 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48788: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33813/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 19:32:30.418414 4035 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48788: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33813/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 19:32:30.420428 4063 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:30.395941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39745 (local address 127.0.0.1:48788)
0810 19:32:30.396675 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:30.396701 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:30.396896 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:30.397506 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:30.397512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:30.397525 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 19:32:30.397936 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:30.397949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:30.399166 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:30.399172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:30.400973 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:30.400989 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:30.401797 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:30.401804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:30.401999 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:30.402082 (+ 83us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 19:32:30.418001 (+ 15919us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33813/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 19:32:30.418410 (+ 409us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.233.190:39745: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33813/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":385,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.233.190:39745: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33813/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 19:32:30.422830 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 3962
I20260810 19:32:30.424086 4027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:30.521502 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 3962
2026-08-10T19:32:30Z chronyd exiting
I20260810 19:32:30.574934 30630 test_util.cc:175] -----------------------------------------------
I20260810 19:32:30.575071 30630 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786390305122638-30630-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (798 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 19:32:30.578806 30630 mini_oidc.cc:152] Starting JWKS server
I20260810 19:32:30.580715 30630 webserver.cc:460] Webserver started at https://127.0.0.1:34371/ using document root <none> and password file <none>
I20260810 19:32:30.580979 30630 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 19:32:30.581641 30630 webserver.cc:460] Webserver started at http://127.0.0.1:41117/ using document root <none> and password file <none>
2026-08-10T19:32:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:30Z Disabled control of system clock
I20260810 19:32:30.617018 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35903
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:39689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:35903
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34371/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/testchainca.cert with env {}
W20260810 19:32:30.952167 4072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:30.952455 4072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:30.952618 4072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:30.959841 4072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:30.959944 4072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:30.960055 4072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:30.960137 4072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:30.960218 4072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 19:32:30.960316 4072 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 19:32:30.960376 4072 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34371/jwks.json
I20260810 19:32:30.970368 4072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:35903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34371/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:30.971706 4072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:30.973035 4072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:30.985116 4078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:30.985158 4080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:30.985186 4077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:30.985945 4072 server_base.cc:1034] running on GCE node
I20260810 19:32:30.986773 4072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:30.988116 4072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:30.989388 4072 hybrid_clock.cc:648] HybridClock initialized: now 1786390350989346 us; error 57 us; skew 500 ppm
I20260810 19:32:30.989809 4072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:30.998558 4072 webserver.cc:460] Webserver started at http://127.29.233.190:34845/ using document root <none> and password file <none>
I20260810 19:32:30.999374 4072 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:30.999527 4072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:30.999891 4072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:31.003052 4072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "1f76594323254e95bd662b494cb98cbf"
format_stamp: "Formatted at 2026-08-10 19:32:30 on dist-test-slave-tm1g"
server_key: "5fa8f1b49b54cb5333bfcf12756e34e8"
server_key_iv: "0f53e871be250b090dd62170f8b54d04"
server_key_version: "encryptionkey@0"
I20260810 19:32:31.003929 4072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "1f76594323254e95bd662b494cb98cbf"
format_stamp: "Formatted at 2026-08-10 19:32:30 on dist-test-slave-tm1g"
server_key: "5fa8f1b49b54cb5333bfcf12756e34e8"
server_key_iv: "0f53e871be250b090dd62170f8b54d04"
server_key_version: "encryptionkey@0"
I20260810 19:32:31.009522 4072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:32:31.013162 4086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:31.014791 4072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:31.015012 4072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "1f76594323254e95bd662b494cb98cbf"
format_stamp: "Formatted at 2026-08-10 19:32:30 on dist-test-slave-tm1g"
server_key: "5fa8f1b49b54cb5333bfcf12756e34e8"
server_key_iv: "0f53e871be250b090dd62170f8b54d04"
server_key_version: "encryptionkey@0"
I20260810 19:32:31.015246 4072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:31.040076 4072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:31.041023 4072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:31.041378 4072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:31.061203 4072 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:35903
I20260810 19:32:31.061228 4141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:35903 every 8 connection(s)
I20260810 19:32:31.062645 4072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:31.066695 4142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:31.069482 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4072
I20260810 19:32:31.069692 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:31.070427 30630 external_mini_cluster.cc:1428] Setting key 7582db9eb17ee1791995e5385f441ec2
I20260810 19:32:31.078847 4142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf: Bootstrap starting.
I20260810 19:32:31.082755 4142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:31.083950 4142 log.cc:824] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:31.084439 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:31.072363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.29.233.190:35903)
0810 19:32:31.073567 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:31.073585 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:31.073631 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:32:31.075433 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:31.075454 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:31.075941 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:31.076333 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:31.076919 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.078170 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.079542 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.080496 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.080694 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:31.082006 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:31.082015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:31.082028 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:31.082645 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:31.082757 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:31.082789 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:31.082917 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:31.083160 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 19:32:31.083456 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":381,"threads_started":1}
I20260810 19:32:31.087373 4142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf: No bootstrap required, opened a new log
I20260810 19:32:31.088536 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:32:31.095700 4142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76594323254e95bd662b494cb98cbf" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35903 } }
I20260810 19:32:31.096097 4142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:31.096220 4142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f76594323254e95bd662b494cb98cbf, State: Initialized, Role: FOLLOWER
I20260810 19:32:31.096822 4142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76594323254e95bd662b494cb98cbf" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35903 } }
I20260810 19:32:31.097064 4142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:31.097213 4142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:31.097384 4142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:31.099858 4142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76594323254e95bd662b494cb98cbf" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35903 } }
I20260810 19:32:31.100379 4142 leader_election.cc:302] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f76594323254e95bd662b494cb98cbf; no voters:
I20260810 19:32:31.101028 4142 leader_election.cc:288] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:31.101213 4147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:31.101969 4147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [term 1 LEADER]: Becoming Leader. State: Replica: 1f76594323254e95bd662b494cb98cbf, State: Running, Role: LEADER
I20260810 19:32:31.102582 4147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76594323254e95bd662b494cb98cbf" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35903 } }
I20260810 19:32:31.103253 4142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:31.105525 4148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f76594323254e95bd662b494cb98cbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76594323254e95bd662b494cb98cbf" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35903 } } }
I20260810 19:32:31.105990 4148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:31.106449 4149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f76594323254e95bd662b494cb98cbf. Latest consensus state: current_term: 1 leader_uuid: "1f76594323254e95bd662b494cb98cbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f76594323254e95bd662b494cb98cbf" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35903 } } }
I20260810 19:32:31.106874 4149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:31.107589 4153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:31.112581 4153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:31.122236 4153 catalog_manager.cc:1347] Generated new cluster ID: 8bd65fc4a602414dbe68ac9aacf1fb5e
I20260810 19:32:31.122413 4153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:31.133124 4153 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:31.134970 4153 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:31.152704 4153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f76594323254e95bd662b494cb98cbf: Generated new TSK 0
I20260810 19:32:31.153839 4153 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 19:32:31.194814 4172 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 19:32:31.194962 4172 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 19:32:31.196110 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:31.168831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.29.233.190:35903)
0810 19:32:31.168996 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:31.169001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:31.169414 (+ 413us) server_negotiation.cc:407] Connection header received
0810 19:32:31.169681 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:31.169685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:31.169760 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:31.169920 (+ 160us) server_negotiation.cc:226] Negotiated authn=JWT
0810 19:32:31.170398 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.171134 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.172511 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.172975 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.173122 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:31.173607 (+ 485us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 19:32:31.195358 (+ 21751us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 19:32:31.195788 (+ 430us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 19:32:31.196254 4145 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 19:32:31.196386 4171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:31.168705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:35903 (local address 127.0.0.1:50142)
0810 19:32:31.169313 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:31.169328 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:31.169496 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:31.169932 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:31.169937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:31.169949 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0810 19:32:31.170227 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:31.170236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.171285 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.171290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:31.172357 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:31.172368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.173164 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.173171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:31.173327 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:31.173360 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 19:32:31.195646 (+ 22286us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 19:32:31.195958 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.233.190:35903: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":347,"threads_started":1}
I20260810 19:32:31.197809 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4072
I20260810 19:32:31.199119 4136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:31.292702 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4072
2026-08-10T19:32:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (772 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 19:32:31.351924 30630 mini_oidc.cc:152] Starting JWKS server
I20260810 19:32:31.353866 30630 webserver.cc:460] Webserver started at https://127.0.0.1:35877/ using document root <none> and password file <none>
I20260810 19:32:31.354153 30630 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 19:32:31.354919 30630 webserver.cc:460] Webserver started at http://127.0.0.1:43385/ using document root <none> and password file <none>
2026-08-10T19:32:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:31Z Disabled control of system clock
I20260810 19:32:31.391705 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42787
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:42787
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35877/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 19:32:31.723059 4179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:31.723438 4179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:31.723606 4179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:31.730886 4179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:31.731051 4179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:31.731148 4179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:31.731297 4179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:31.731388 4179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 19:32:31.731480 4179 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 19:32:31.731541 4179 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35877/jwks.json
I20260810 19:32:31.741984 4179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:42787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35877/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:31.743903 4179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:31.746569 4179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:31.758989 4185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:31.759057 4188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:31.759310 4186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:31.760645 4179 server_base.cc:1034] running on GCE node
I20260810 19:32:31.761564 4179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:31.763159 4179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:31.764395 4179 hybrid_clock.cc:648] HybridClock initialized: now 1786390351764351 us; error 38 us; skew 500 ppm
I20260810 19:32:31.764757 4179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:31.767244 4179 webserver.cc:460] Webserver started at http://127.29.233.190:33693/ using document root <none> and password file <none>
I20260810 19:32:31.767995 4179 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:31.768142 4179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:31.768492 4179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:31.772020 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "9a9927511f1748e7afbbd4e7691741d8"
format_stamp: "Formatted at 2026-08-10 19:32:31 on dist-test-slave-tm1g"
server_key: "852a12a889d6dcc94268b90acbdd001d"
server_key_iv: "a0c39cd3e548e3d2e0e3a91a85140c7c"
server_key_version: "encryptionkey@0"
I20260810 19:32:31.772868 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "9a9927511f1748e7afbbd4e7691741d8"
format_stamp: "Formatted at 2026-08-10 19:32:31 on dist-test-slave-tm1g"
server_key: "852a12a889d6dcc94268b90acbdd001d"
server_key_iv: "a0c39cd3e548e3d2e0e3a91a85140c7c"
server_key_version: "encryptionkey@0"
I20260810 19:32:31.778242 4179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260810 19:32:31.781652 4194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:31.783020 4179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:31.783228 4179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "9a9927511f1748e7afbbd4e7691741d8"
format_stamp: "Formatted at 2026-08-10 19:32:31 on dist-test-slave-tm1g"
server_key: "852a12a889d6dcc94268b90acbdd001d"
server_key_iv: "a0c39cd3e548e3d2e0e3a91a85140c7c"
server_key_version: "encryptionkey@0"
I20260810 19:32:31.783456 4179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:31.810174 4179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:31.811141 4179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:31.811484 4179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:31.831724 4179 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:42787
I20260810 19:32:31.831732 4251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:42787 every 8 connection(s)
I20260810 19:32:31.833137 4179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:31.834785 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4179
I20260810 19:32:31.835029 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:31.835891 30630 external_mini_cluster.cc:1428] Setting key af003882a3fcf6e368429320e1f72a37
I20260810 19:32:31.837939 4252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:31.849469 4252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8: Bootstrap starting.
I20260810 19:32:31.850238 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:31.837800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.29.233.190:42787)
0810 19:32:31.838844 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:31.838855 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:31.838895 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:32:31.840490 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:31.840518 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:31.840964 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:31.841398 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:31.841953 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.843390 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.844727 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.845408 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.845605 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:31.847446 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:31.847457 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:31.847471 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:31.848228 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:31.848305 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:31.848359 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:31.848592 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:31.848824 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 19:32:31.849152 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":418,"threads_started":1}
I20260810 19:32:31.852751 4252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:31.853871 4252 log.cc:824] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:31.854188 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:32:31.857117 4252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8: No bootstrap required, opened a new log
I20260810 19:32:31.864265 4252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9927511f1748e7afbbd4e7691741d8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42787 } }
I20260810 19:32:31.864646 4252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:31.864755 4252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9927511f1748e7afbbd4e7691741d8, State: Initialized, Role: FOLLOWER
I20260810 19:32:31.865387 4252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9927511f1748e7afbbd4e7691741d8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42787 } }
I20260810 19:32:31.865657 4252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:31.865813 4252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:31.865999 4252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:31.868444 4252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9927511f1748e7afbbd4e7691741d8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42787 } }
I20260810 19:32:31.868939 4252 leader_election.cc:302] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a9927511f1748e7afbbd4e7691741d8; no voters:
I20260810 19:32:31.869522 4252 leader_election.cc:288] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:31.869709 4257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:31.870483 4257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [term 1 LEADER]: Becoming Leader. State: Replica: 9a9927511f1748e7afbbd4e7691741d8, State: Running, Role: LEADER
I20260810 19:32:31.871068 4257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9927511f1748e7afbbd4e7691741d8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42787 } }
I20260810 19:32:31.871560 4252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:31.873728 4259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9927511f1748e7afbbd4e7691741d8. Latest consensus state: current_term: 1 leader_uuid: "9a9927511f1748e7afbbd4e7691741d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9927511f1748e7afbbd4e7691741d8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42787 } } }
I20260810 19:32:31.874084 4259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:31.876696 4258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a9927511f1748e7afbbd4e7691741d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9927511f1748e7afbbd4e7691741d8" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42787 } } }
I20260810 19:32:31.877144 4258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:31.877866 4264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:31.882859 4264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:31.891279 4264 catalog_manager.cc:1347] Generated new cluster ID: 5ea0ad1966b74c3fb1adf2ce15e52b06
I20260810 19:32:31.891392 4264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:31.907284 4264 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:31.908787 4264 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:31.917001 4264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a9927511f1748e7afbbd4e7691741d8: Generated new TSK 0
I20260810 19:32:31.918105 4264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:32:31.950106 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:31.941185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45184 (local address 127.29.233.190:42787)
0810 19:32:31.941394 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:31.941401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:31.941786 (+ 385us) server_negotiation.cc:407] Connection header received
0810 19:32:31.942004 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:31.942010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:31.942114 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:31.942280 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0810 19:32:31.942802 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.943982 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.945361 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.946245 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.946460 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:31.947026 (+ 566us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 19:32:31.949409 (+ 2383us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 19:32:31.949771 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 19:32:31.950248 4255 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 19:32:31.951797 4283 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:31.941022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42787 (local address 127.0.0.1:45184)
0810 19:32:31.941682 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:31.941699 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:31.941820 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:31.942295 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:31.942298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:31.942357 (+ 59us) client_negotiation.cc:189] Negotiated authn=JWT
0810 19:32:31.942652 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:31.942661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.944129 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.944132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:31.945210 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:31.945221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:31.946502 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:31.946509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:31.946695 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:31.946732 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 19:32:31.949626 (+ 2894us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 19:32:31.951376 (+ 1750us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.233.190:42787: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":1440,"thread_start_us":378,"threads_started":1}
I20260810 19:32:31.952965 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4179
I20260810 19:32:31.954049 4246 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 19:32:31.961833 4284 webserver.cc:533] Webserver: sslize failed:
E20260810 19:32:31.961959 4284 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 19:32:32.047129 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4179
2026-08-10T19:32:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (754 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T19:32:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:32Z Disabled control of system clock
I20260810 19:32:32.142589 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43217
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:43217
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/testchainca.cert with env {}
W20260810 19:32:32.479118 4289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:32.479408 4289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:32.479552 4289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:32.486801 4289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:32.486899 4289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:32.487002 4289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:32.487095 4289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:32.487183 4289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 19:32:32.487277 4289 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 19:32:32.487337 4289 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 19:32:32.497201 4289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:43217
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:32.498564 4289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:32.499888 4289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:32.512022 4294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:32.512141 4295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:32.512145 4297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:32.513024 4289 server_base.cc:1034] running on GCE node
I20260810 19:32:32.513835 4289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:32.515341 4289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:32.516553 4289 hybrid_clock.cc:648] HybridClock initialized: now 1786390352516504 us; error 32 us; skew 500 ppm
I20260810 19:32:32.516902 4289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:32:32.519366 4289 webserver.cc:460] Webserver started at http://127.29.233.190:39047/ using document root <none> and password file <none>
I20260810 19:32:32.520085 4289 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:32.520206 4289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:32.520560 4289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:32.523476 4289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "98066997421a47babb6e1b8d0272f805"
format_stamp: "Formatted at 2026-08-10 19:32:32 on dist-test-slave-tm1g"
server_key: "aa4b2bedaf742d710ff48be3aec64d90"
server_key_iv: "886d832f6efca07f853e8ccb451db70e"
server_key_version: "encryptionkey@0"
I20260810 19:32:32.524302 4289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "98066997421a47babb6e1b8d0272f805"
format_stamp: "Formatted at 2026-08-10 19:32:32 on dist-test-slave-tm1g"
server_key: "aa4b2bedaf742d710ff48be3aec64d90"
server_key_iv: "886d832f6efca07f853e8ccb451db70e"
server_key_version: "encryptionkey@0"
I20260810 19:32:32.529460 4289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:32:32.533175 4304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:32.534919 4289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:32.535113 4289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "98066997421a47babb6e1b8d0272f805"
format_stamp: "Formatted at 2026-08-10 19:32:32 on dist-test-slave-tm1g"
server_key: "aa4b2bedaf742d710ff48be3aec64d90"
server_key_iv: "886d832f6efca07f853e8ccb451db70e"
server_key_version: "encryptionkey@0"
I20260810 19:32:32.535341 4289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:32.557199 4289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:32.558094 4289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:32.558421 4289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:32.578267 4289 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:43217
I20260810 19:32:32.578331 4360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:43217 every 8 connection(s)
I20260810 19:32:32.579695 4289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:32.583652 4361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:32.584174 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4289
I20260810 19:32:32.584383 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:32.585062 30630 external_mini_cluster.cc:1428] Setting key 806101c7855e075b25dea1c984ec67ba
I20260810 19:32:32.595995 4361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805: Bootstrap starting.
I20260810 19:32:32.598433 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:32.587012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.29.233.190:43217)
0810 19:32:32.587985 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:32.587996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:32.588042 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:32:32.589836 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:32.589860 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:32.590291 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:32.590737 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:32.591262 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:32.592529 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:32.593836 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:32.594598 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:32.594794 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:32.596074 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:32.596081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:32.596094 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:32:32.596656 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:32.596732 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:32:32.596757 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:32.596886 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:32.597130 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 19:32:32.597465 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":330,"threads_started":1}
I20260810 19:32:32.599540 4361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:32.600875 4361 log.cc:824] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:32.602607 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:32:32.603888 4361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805: No bootstrap required, opened a new log
I20260810 19:32:32.611840 4361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98066997421a47babb6e1b8d0272f805" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43217 } }
I20260810 19:32:32.612285 4361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:32.612411 4361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98066997421a47babb6e1b8d0272f805, State: Initialized, Role: FOLLOWER
I20260810 19:32:32.612947 4361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98066997421a47babb6e1b8d0272f805" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43217 } }
I20260810 19:32:32.613195 4361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:32.613344 4361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:32.613529 4361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:32.615732 4361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98066997421a47babb6e1b8d0272f805" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43217 } }
I20260810 19:32:32.616233 4361 leader_election.cc:302] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98066997421a47babb6e1b8d0272f805; no voters:
I20260810 19:32:32.616827 4361 leader_election.cc:288] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:32.617000 4366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:32.617739 4366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [term 1 LEADER]: Becoming Leader. State: Replica: 98066997421a47babb6e1b8d0272f805, State: Running, Role: LEADER
I20260810 19:32:32.618264 4366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98066997421a47babb6e1b8d0272f805" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43217 } }
I20260810 19:32:32.618980 4361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:32.621058 4367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98066997421a47babb6e1b8d0272f805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98066997421a47babb6e1b8d0272f805" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43217 } } }
I20260810 19:32:32.621399 4367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:32.621881 4368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98066997421a47babb6e1b8d0272f805. Latest consensus state: current_term: 1 leader_uuid: "98066997421a47babb6e1b8d0272f805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98066997421a47babb6e1b8d0272f805" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43217 } } }
I20260810 19:32:32.622236 4368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:32.623994 4371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:32.628587 4371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:32.637311 4371 catalog_manager.cc:1347] Generated new cluster ID: 8da36366470845bb94568994d58acab7
I20260810 19:32:32.637444 4371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:32.649806 4371 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:32.651641 4371 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:32.669145 4371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98066997421a47babb6e1b8d0272f805: Generated new TSK 0
I20260810 19:32:32.670262 4371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:32:32.756228 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:32.694747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.29.233.190:43217)
0810 19:32:32.694940 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:32.694947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:32.695469 (+ 522us) server_negotiation.cc:407] Connection header received
0810 19:32:32.695688 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:32.695695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:32.695782 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:32.695951 (+ 169us) server_negotiation.cc:226] Negotiated authn=JWT
0810 19:32:32.696398 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:32.697555 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:32.699037 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:32.699743 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:32.699867 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:32.700339 (+ 472us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 19:32:32.755389 (+ 55050us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 19:32:32.755859 (+ 470us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60130: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 19:32:32.756399 4364 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60130: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 19:32:32.756480 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:32.694731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:43217 (local address 127.0.0.1:60130)
0810 19:32:32.695372 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:32.695388 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:32.695507 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:32.695987 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:32.695991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:32.696001 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 19:32:32.696246 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:32.696254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:32.697718 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:32.697722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:32.698900 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:32.698912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:32.699915 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:32.699919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:32.700050 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:32.700083 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 19:32:32.755746 (+ 55663us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 19:32:32.756060 (+ 314us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.233.190:43217: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":338,"threads_started":1}
I20260810 19:32:32.757716 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4289
I20260810 19:32:32.759063 4356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:32.859848 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4289
2026-08-10T19:32:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (792 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (425 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (417 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:33 dist-test-slave-tm1g krb5kdc[4408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:33 dist-test-slave-tm1g krb5kdc[4408](info): set up 2 sockets
Aug 10 19:32:33 dist-test-slave-tm1g krb5kdc[4408](info): commencing operation
krb5kdc: starting...
W20260810 19:32:34.866215 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:35 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:35.118804 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39827
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:39827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:35.453102 4423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:35.453454 4423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:35.453619 4423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:35.461395 4423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:35.461508 4423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:35.461599 4423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:35.461688 4423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:35.461750 4423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:35.471990 4423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:39827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:35.473397 4423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:35.474824 4423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:35.487124 4428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:35.487144 4429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:35.487020 4431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:35.487833 4423 server_base.cc:1034] running on GCE node
I20260810 19:32:35.488659 4423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:35.490079 4423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:35.491294 4423 hybrid_clock.cc:648] HybridClock initialized: now 1786390355491250 us; error 36 us; skew 500 ppm
I20260810 19:32:35.491669 4423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:35 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390355, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:35.494840 4423 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:35.496390 4423 webserver.cc:460] Webserver started at http://127.29.233.190:45235/ using document root <none> and password file <none>
I20260810 19:32:35.497105 4423 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:35.497256 4423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:35.497597 4423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:35.500633 4423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "12d8f5407bdb42dfad5def8b098ea9f2"
format_stamp: "Formatted at 2026-08-10 19:32:35 on dist-test-slave-tm1g"
server_key: "6226d7e97699e2ad33dec7561c164689"
server_key_iv: "c35876d705a75cf73b6d6047ba6ed0cb"
server_key_version: "encryptionkey@0"
I20260810 19:32:35.501415 4423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "12d8f5407bdb42dfad5def8b098ea9f2"
format_stamp: "Formatted at 2026-08-10 19:32:35 on dist-test-slave-tm1g"
server_key: "6226d7e97699e2ad33dec7561c164689"
server_key_iv: "c35876d705a75cf73b6d6047ba6ed0cb"
server_key_version: "encryptionkey@0"
I20260810 19:32:35.506567 4423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 19:32:35.509976 4438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:35.511327 4423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:35.511533 4423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "12d8f5407bdb42dfad5def8b098ea9f2"
format_stamp: "Formatted at 2026-08-10 19:32:35 on dist-test-slave-tm1g"
server_key: "6226d7e97699e2ad33dec7561c164689"
server_key_iv: "c35876d705a75cf73b6d6047ba6ed0cb"
server_key_version: "encryptionkey@0"
I20260810 19:32:35.511734 4423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:35.529867 4423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:35.534262 4423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:35.534590 4423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:35.553315 4423 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:39827
I20260810 19:32:35.553334 4490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:39827 every 8 connection(s)
I20260810 19:32:35.554723 4423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:35.558364 4491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:35.562930 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4423
I20260810 19:32:35.563143 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:35.563998 30630 external_mini_cluster.cc:1428] Setting key 480cfdc35cb3c88719f4ed7c363c6ca3
I20260810 19:32:35.571429 4491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2: Bootstrap starting.
I20260810 19:32:35.575137 4491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:35.576188 4491 log.cc:824] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:35.579107 4491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2: No bootstrap required, opened a new log
Aug 10 19:32:35 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:35.587585 4491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39827 } }
I20260810 19:32:35.588016 4491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:35.588147 4491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d8f5407bdb42dfad5def8b098ea9f2, State: Initialized, Role: FOLLOWER
I20260810 19:32:35.588495 4494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:35.566114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.29.233.190:39827)
0810 19:32:35.566959 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:35.566975 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:35.567024 (+ 49us) server_negotiation.cc:407] Connection header received
0810 19:32:35.569687 (+ 2663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:35.569716 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:35.570195 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:35.570657 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:35.571914 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:35.573954 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:35.575328 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:35.576014 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:35.576212 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:35.580746 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:35.580754 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:35.580768 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:35.580807 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:35.584021 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:35.584795 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:35.584804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:35.584819 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:35.584997 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:35.585426 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:35.585433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:35.585436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:35.586037 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:35.586412 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:35.586862 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 19:32:35.587281 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":352,"threads_started":1}
I20260810 19:32:35.588815 4491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39827 } }
I20260810 19:32:35.589107 4491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:35.589263 4491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:35.589460 4491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:35.592052 4491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39827 } }
I20260810 19:32:35.592547 4491 leader_election.cc:302] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d8f5407bdb42dfad5def8b098ea9f2; no voters:
I20260810 19:32:35.593118 4491 leader_election.cc:288] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:35.593286 4496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:35.593979 4496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [term 1 LEADER]: Becoming Leader. State: Replica: 12d8f5407bdb42dfad5def8b098ea9f2, State: Running, Role: LEADER
I20260810 19:32:35.594553 4496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39827 } }
I20260810 19:32:35.595252 4491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:35.597697 4498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d8f5407bdb42dfad5def8b098ea9f2. Latest consensus state: current_term: 1 leader_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39827 } } }
I20260810 19:32:35.598027 4498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:35.598471 4497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d8f5407bdb42dfad5def8b098ea9f2" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39827 } } }
I20260810 19:32:35.598902 4497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:35.602972 4503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:35.608064 4503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:35.616663 4503 catalog_manager.cc:1347] Generated new cluster ID: 826f73b2f1064c08aea2a2e56088f1b5
I20260810 19:32:35.616775 4503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:35.637311 4503 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:35.638674 4503 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:35.650180 4503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d8f5407bdb42dfad5def8b098ea9f2: Generated new TSK 0
I20260810 19:32:35.651034 4503 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:32:35.712520 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39827
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:36.054726 4517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:36.055123 4517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:36.055301 4517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:36.062578 4517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:36.062691 4517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:36.062837 4517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:36.072680 4517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:36.074009 4517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:36.075409 4517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:36.087971 4523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:36.088574 4525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:36.088939 4527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:36.089917 4517 server_base.cc:1034] running on GCE node
I20260810 19:32:36.090562 4517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:36.091401 4517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:36.092633 4517 hybrid_clock.cc:648] HybridClock initialized: now 1786390356092598 us; error 52 us; skew 500 ppm
I20260810 19:32:36.092937 4517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:36 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390356, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:36.100487 4517 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:32:36.102898 4517 webserver.cc:460] Webserver started at http://127.29.233.129:35959/ using document root <none> and password file <none>
I20260810 19:32:36.103893 4517 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:36.104035 4517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:36.104455 4517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:36.108816 4517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "d193ffc9750b4d968108524990458df6"
format_stamp: "Formatted at 2026-08-10 19:32:36 on dist-test-slave-tm1g"
server_key: "6740520769c7107fa0f4a06c0b648d8c"
server_key_iv: "3454a6546459ff1de1ff35e96fdc52dc"
server_key_version: "encryptionkey@0"
I20260810 19:32:36.109977 4517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "d193ffc9750b4d968108524990458df6"
format_stamp: "Formatted at 2026-08-10 19:32:36 on dist-test-slave-tm1g"
server_key: "6740520769c7107fa0f4a06c0b648d8c"
server_key_iv: "3454a6546459ff1de1ff35e96fdc52dc"
server_key_version: "encryptionkey@0"
I20260810 19:32:36.117228 4517 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260810 19:32:36.121694 4535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:36.123517 4517 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 19:32:36.123703 4517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "d193ffc9750b4d968108524990458df6"
format_stamp: "Formatted at 2026-08-10 19:32:36 on dist-test-slave-tm1g"
server_key: "6740520769c7107fa0f4a06c0b648d8c"
server_key_iv: "3454a6546459ff1de1ff35e96fdc52dc"
server_key_version: "encryptionkey@0"
I20260810 19:32:36.123972 4517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:36.149505 4517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:36.154295 4517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:36.154691 4517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:36.155673 4517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:36.157038 4517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:36.157114 4517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:36.157286 4517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:36.157356 4517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:36.190042 4517 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:37539
I20260810 19:32:36.190157 4652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:37539 every 8 connection(s)
I20260810 19:32:36.191479 4517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:36.199918 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4517
I20260810 19:32:36.200131 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:36.200913 30630 external_mini_cluster.cc:1428] Setting key 4d6a782d43ed3a558ade8a46214ea7a6
Aug 10 19:32:36 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390356, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:36.214449 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:36.194200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:51449 (local address 127.29.233.190:39827)
0810 19:32:36.194837 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:36.194848 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:36.195683 (+ 835us) server_negotiation.cc:407] Connection header received
0810 19:32:36.197647 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:36.197652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:36.197746 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:36.197888 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:36.199512 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.200445 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.202484 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.203057 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.203210 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:36.207183 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:36.207192 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:36.207200 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:36.207249 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:36.211070 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:36.211888 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:36.211902 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:36.211908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:36.212062 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:36.212480 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:36.212488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:36.212494 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:36.212882 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:36.213053 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:36.213908 (+ 855us) server_negotiation.cc:299] Negotiation successful
0810 19:32:36.214093 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":413,"threads_started":1}
I20260810 19:32:36.216153 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:36.194691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39827 (local address 127.29.233.129:51449)
0810 19:32:36.195493 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:36.195564 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:36.197367 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:36.198137 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:36.198145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:36.198730 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:36.199297 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:36.199314 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.200609 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.200614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:36.202280 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:36.202328 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.203315 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.203320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:36.203544 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:36.204291 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:36.204319 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:36.206929 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:36.211263 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:36.211271 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:36.211285 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:36.211742 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:36.212170 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:36.212173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:36.212175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:36.212362 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:36.213010 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:36.213015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:36.213435 (+ 420us) client_negotiation.cc:769] Sending connection context
0810 19:32:36.213790 (+ 355us) client_negotiation.cc:240] Negotiation successful
0810 19:32:36.214449 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":288,"threads_started":1}
I20260810 19:32:36.217824 4653 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39827
I20260810 19:32:36.218148 4653 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:36.218954 4653 heartbeater.cc:511] Master 127.29.233.190:39827 requested a full tablet report, sending...
I20260810 19:32:36.221079 4456 ts_manager.cc:194] Registered new tserver with Master: d193ffc9750b4d968108524990458df6 (127.29.233.129:37539)
I20260810 19:32:36.222642 4456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:51449
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:32:36.262485 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39827
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:36.601639 4660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:36.601955 4660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:36.602133 4660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:36.609346 4660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:36.609512 4660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:36.609691 4660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:36.620254 4660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:36.621587 4660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:36.623013 4660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:36.636340 4666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:36.636305 4667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:36.636390 4669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:36.637270 4660 server_base.cc:1034] running on GCE node
I20260810 19:32:36.637923 4660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:36.638886 4660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:36.640055 4660 hybrid_clock.cc:648] HybridClock initialized: now 1786390356639997 us; error 52 us; skew 500 ppm
I20260810 19:32:36.640373 4660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:36 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390356, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:36.643545 4660 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:32:36.644932 4660 webserver.cc:460] Webserver started at http://127.29.233.130:34299/ using document root <none> and password file <none>
I20260810 19:32:36.645650 4660 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:36.645797 4660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:36.646119 4660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:36.649058 4660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "99df57417f7249abbf64a6ab6e631804"
format_stamp: "Formatted at 2026-08-10 19:32:36 on dist-test-slave-tm1g"
server_key: "f08faf8140dd971a442d560a8405e9dd"
server_key_iv: "be3cd81d3e0eca84fa14f4a0cce06fc7"
server_key_version: "encryptionkey@0"
I20260810 19:32:36.649844 4660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "99df57417f7249abbf64a6ab6e631804"
format_stamp: "Formatted at 2026-08-10 19:32:36 on dist-test-slave-tm1g"
server_key: "f08faf8140dd971a442d560a8405e9dd"
server_key_iv: "be3cd81d3e0eca84fa14f4a0cce06fc7"
server_key_version: "encryptionkey@0"
I20260810 19:32:36.655063 4660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 19:32:36.658435 4679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:36.659710 4660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:36.659888 4660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "99df57417f7249abbf64a6ab6e631804"
format_stamp: "Formatted at 2026-08-10 19:32:36 on dist-test-slave-tm1g"
server_key: "f08faf8140dd971a442d560a8405e9dd"
server_key_iv: "be3cd81d3e0eca84fa14f4a0cce06fc7"
server_key_version: "encryptionkey@0"
I20260810 19:32:36.660101 4660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:36.680984 4660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:36.685377 4660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:36.685669 4660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:36.686702 4660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:36.688297 4660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:36.688400 4660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:36.688555 4660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:36.688627 4660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:36.722746 4660 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:41571
I20260810 19:32:36.722832 4799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:41571 every 8 connection(s)
I20260810 19:32:36.724085 4660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:36.728960 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4660
I20260810 19:32:36.729166 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:36.729744 30630 external_mini_cluster.cc:1428] Setting key daa585ab6af7bd306e077c20ae2fc3f7
Aug 10 19:32:36 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390356, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:36.748170 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:36.726876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:39119 (local address 127.29.233.190:39827)
0810 19:32:36.727368 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:36.727373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:36.728445 (+ 1072us) server_negotiation.cc:407] Connection header received
0810 19:32:36.731114 (+ 2669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:36.731119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:36.731210 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:36.731333 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:36.733491 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.734369 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.736915 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.737378 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.737481 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:36.742010 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:36.742016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:36.742019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:36.742053 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:36.744852 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:36.745621 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:36.745628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:36.745632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:36.745769 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:36.746240 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:36.746246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:36.746249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:36.746638 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:36.746802 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:36.747511 (+ 709us) server_negotiation.cc:299] Negotiation successful
0810 19:32:36.747757 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":238,"threads_started":1}
I20260810 19:32:36.749336 4803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:36.727491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39827 (local address 127.29.233.130:39119)
0810 19:32:36.728265 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:36.728312 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:36.730790 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:36.731615 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:36.731627 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:36.732365 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:36.733231 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:36.733261 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.734550 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.734557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:36.736738 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:36.736757 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:36.737627 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:36.737636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:36.737978 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:36.738750 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:36.738792 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:36.741741 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:36.745004 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:36.745014 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:36.745033 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:36.745487 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:36.745892 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:36.745897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:36.745900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:36.746111 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:36.746776 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:36.746785 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:36.747148 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 19:32:36.747430 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 19:32:36.748009 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":332,"threads_started":1}
I20260810 19:32:36.751001 4800 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39827
I20260810 19:32:36.751366 4800 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:36.752329 4800 heartbeater.cc:511] Master 127.29.233.190:39827 requested a full tablet report, sending...
I20260810 19:32:36.754228 4456 ts_manager.cc:194] Registered new tserver with Master: 99df57417f7249abbf64a6ab6e631804 (127.29.233.130:41571)
I20260810 19:32:36.755568 4456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:39119
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:32:36.789945 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39827
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:37.127195 4806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:37.127544 4806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:37.127722 4806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:37.135290 4806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:37.135468 4806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:37.135651 4806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:37.145340 4806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:37.146759 4806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:37.148136 4806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:37.161581 4812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:37.161574 4817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:37.162379 4813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:37.162712 4806 server_base.cc:1034] running on GCE node
I20260810 19:32:37.163450 4806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:37.164589 4806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:37.165807 4806 hybrid_clock.cc:648] HybridClock initialized: now 1786390357165755 us; error 45 us; skew 500 ppm
I20260810 19:32:37.166100 4806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.169417 4806 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:32:37.171046 4806 webserver.cc:460] Webserver started at http://127.29.233.131:37159/ using document root <none> and password file <none>
I20260810 19:32:37.171816 4806 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:37.171924 4806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:37.172194 4806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:37.175323 4806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "7db268453022479ba781f606762182f7"
format_stamp: "Formatted at 2026-08-10 19:32:37 on dist-test-slave-tm1g"
server_key: "50a3598c6c4962310c4bafe415656730"
server_key_iv: "cf4faca3d06bf4750902a6bfa7e3b3f3"
server_key_version: "encryptionkey@0"
I20260810 19:32:37.176092 4806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "7db268453022479ba781f606762182f7"
format_stamp: "Formatted at 2026-08-10 19:32:37 on dist-test-slave-tm1g"
server_key: "50a3598c6c4962310c4bafe415656730"
server_key_iv: "cf4faca3d06bf4750902a6bfa7e3b3f3"
server_key_version: "encryptionkey@0"
I20260810 19:32:37.181519 4806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:32:37.185309 4825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:37.186816 4806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:37.186961 4806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "7db268453022479ba781f606762182f7"
format_stamp: "Formatted at 2026-08-10 19:32:37 on dist-test-slave-tm1g"
server_key: "50a3598c6c4962310c4bafe415656730"
server_key_iv: "cf4faca3d06bf4750902a6bfa7e3b3f3"
server_key_version: "encryptionkey@0"
I20260810 19:32:37.187137 4806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:37.204053 4806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:37.208447 4806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:37.208694 4806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:37.209605 4806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:37.210999 4806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:37.211073 4806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:37.211159 4806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:37.211203 4806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:37.226017 4653 heartbeater.cc:503] Master 127.29.233.190:39827 was elected leader, sending a full tablet report...
I20260810 19:32:37.245584 4806 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:43453
I20260810 19:32:37.245661 4944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:43453 every 8 connection(s)
I20260810 19:32:37.246995 4806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:37.255578 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4806
I20260810 19:32:37.255792 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:37.256250 30630 external_mini_cluster.cc:1428] Setting key 7a8973a64663481b266185ce3f4f4d1a
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:37.268039 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.249840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:37599 (local address 127.29.233.190:39827)
0810 19:32:37.250418 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.250423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.251342 (+ 919us) server_negotiation.cc:407] Connection header received
0810 19:32:37.253196 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.253201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.253292 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.253416 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.254995 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.255852 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.257885 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.258382 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.258512 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.262002 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:37.262009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:37.262023 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:37.262064 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:37.264850 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.265596 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.265603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.265607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.265739 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.266184 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.266190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.266194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.266543 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:37.266711 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:37.267399 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 19:32:37.267626 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":278,"threads_started":1}
I20260810 19:32:37.269002 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.250368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39827 (local address 127.29.233.131:37599)
0810 19:32:37.251170 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:37.251218 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:37.252916 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:37.253659 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:37.253667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:37.254171 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:37.254815 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:37.254833 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.256049 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.256053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:37.257745 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:37.257757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.258627 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.258635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:37.258878 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.259590 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:37.259612 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:37.261786 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:37.264992 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:37.265002 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:37.265019 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:37.265460 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:37.265855 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:37.265861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:37.265865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:37.266067 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:37.266676 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:37.266684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:37.267034 (+ 350us) client_negotiation.cc:769] Sending connection context
0810 19:32:37.267313 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 19:32:37.267868 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":305,"threads_started":1}
I20260810 19:32:37.270464 4945 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39827
I20260810 19:32:37.270781 4945 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:37.271530 4945 heartbeater.cc:511] Master 127.29.233.190:39827 requested a full tablet report, sending...
I20260810 19:32:37.272933 4456 ts_manager.cc:194] Registered new tserver with Master: 7db268453022479ba781f606762182f7 (127.29.233.131:43453)
I20260810 19:32:37.273691 4456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:37599
I20260810 19:32:37.284008 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/test-admin.keytab.
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.318077 30630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 19:32:37.323834 4954 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:37.321085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33298 (local address 127.29.233.129:37539)
0810 19:32:37.321791 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.321798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.321824 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:32:37.321882 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.321889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.322107 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.322299 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.323309 (+ 1010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33298: BlockingRecv error: recv got EOF from 127.0.0.1:33298 (error 108)
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":348,"threads_started":1}
W20260810 19:32:37.323916 30630 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc' of type FILE
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.325811 30630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.327301 30630 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:37.344255 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.329049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.29.233.129:37539)
0810 19:32:37.329280 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.329288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.329543 (+ 255us) server_negotiation.cc:407] Connection header received
0810 19:32:37.329723 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.329729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.329797 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.329907 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.331034 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.332029 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.333261 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.334153 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.334288 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.338541 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:37.338554 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:37.338570 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:37.338605 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:37.341558 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.342160 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.342167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.342169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.342290 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.342723 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.342727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.342729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.343135 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:37.343368 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:37.343643 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 19:32:37.343946 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 19:32:37.350371 4954 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:37.348413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.29.233.129:37539)
0810 19:32:37.348640 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.348647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.348945 (+ 298us) server_negotiation.cc:407] Connection header received
0810 19:32:37.349137 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.349141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.349196 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.349300 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.350086 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33318: BlockingRecv error: recv got EOF from 127.0.0.1:33318 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 19:32:37.350613 30630 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc' of type FILE
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.352221 30630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.353543 30630 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:37.368307 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.354905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.29.233.129:37539)
0810 19:32:37.355110 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.355117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.355470 (+ 353us) server_negotiation.cc:407] Connection header received
0810 19:32:37.355658 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.355670 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.355736 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.355859 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.356821 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.357515 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.358798 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.359241 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.359380 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.363031 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:37.363041 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:37.363047 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:37.363104 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:37.365710 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.366412 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.366418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.366421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.366549 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.366947 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.366952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.366957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.367227 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:37.367507 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:37.367793 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 19:32:37.368009 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 19:32:37.372686 4954 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:37.371082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33338 (local address 127.29.233.129:37539)
0810 19:32:37.371297 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.371301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.371412 (+ 111us) server_negotiation.cc:407] Connection header received
0810 19:32:37.371555 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.371563 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.371653 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.371759 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.372439 (+ 680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33338: BlockingRecv error: recv got EOF from 127.0.0.1:33338 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260810 19:32:37.373131 30630 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786390305122638-30630-0/krb5kdc/krb5cc)
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.374688 30630 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:37.375993 30630 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:37.389917 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.377403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.29.233.129:37539)
0810 19:32:37.377633 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.377638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.377712 (+ 74us) server_negotiation.cc:407] Connection header received
0810 19:32:37.377831 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.377835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.377888 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.377963 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.378926 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.379648 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.380833 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.381256 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.381348 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.384862 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:37.384866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:37.384869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:37.384900 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:37.387504 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.388108 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.388111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.388113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.388256 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.388570 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.388573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.388575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.388814 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:37.389046 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:37.389395 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 19:32:37.389609 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:32:37.391661 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4517
I20260810 19:32:37.403519 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.392333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33352 (local address 127.29.233.129:37539)
0810 19:32:37.392541 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.392546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.392778 (+ 232us) server_negotiation.cc:407] Connection header received
0810 19:32:37.392895 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.392898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.392950 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.393032 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.393848 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.394589 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.395848 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.396324 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.396422 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.398530 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:37.398534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:37.398537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:37.398564 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:37.400882 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.401606 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.401610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.401612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.401732 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.402171 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.402174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.402176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.402483 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:37.402679 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:37.402998 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 19:32:37.403156 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:32:37.404163 4648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:37.489146 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4517
I20260810 19:32:37.508733 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4660
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:32:37.524607 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.509579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.29.233.130:41571)
0810 19:32:37.510126 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.510133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.510159 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:32:37.510221 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.510227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.510514 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.510696 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.511759 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.512820 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.514078 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.514904 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.515063 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.519024 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:37.519035 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:37.519049 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:37.519090 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:37.521998 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.522809 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.522816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.522818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.522938 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.523276 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.523279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.523281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.523616 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:37.523844 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:37.524104 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 19:32:37.524303 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":273,"threads_started":1}
I20260810 19:32:37.525924 4795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:37.608565 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4660
I20260810 19:32:37.627439 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4806
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:32:37.643241 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:37.628492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40102 (local address 127.29.233.131:43453)
0810 19:32:37.629198 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:37.629206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:37.629234 (+ 28us) server_negotiation.cc:407] Connection header received
0810 19:32:37.629299 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:37.629306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:37.629539 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:37.629670 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:37.630793 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.631841 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.632980 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:37.633520 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:37.633614 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:37.637614 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:37.637626 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:37.637640 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:37.637673 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:37.640726 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.641343 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.641350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.641353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.641471 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:37.641801 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:37.641804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:37.641806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:37.642133 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:37.642427 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:37.642662 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 19:32:37.642919 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":384,"threads_started":1}
I20260810 19:32:37.644605 4940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:37.729370 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4806
I20260810 19:32:37.752158 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4423
I20260810 19:32:37.753459 4486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:37.864485 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4423
2026-08-10T19:32:37Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4169 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4976](info): set up 2 sockets
Aug 10 19:32:37 dist-test-slave-tm1g krb5kdc[4976](info): commencing operation
krb5kdc: starting...
W20260810 19:32:39.030400 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:39 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab.
Entry for principal oryx/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab.
I20260810 19:32:39.302064 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:46487
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab
--principal=oryx/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:39.646000 4989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:39.646284 4989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:39.646447 4989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:39.653934 4989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:39.654042 4989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:39.654088 4989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:39.654137 4989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:39.654182 4989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:39.664114 4989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab
--principal=oryx/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:46487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:39.665449 4989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:39.666906 4989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:39.678903 4996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:39.678833 4997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:39.678800 4999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:39.680063 4989 server_base.cc:1034] running on GCE node
I20260810 19:32:39.680891 4989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:39.682436 4989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:39.683676 4989 hybrid_clock.cc:648] HybridClock initialized: now 1786390359683629 us; error 53 us; skew 500 ppm
I20260810 19:32:39.684037 4989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:39 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:39.687199 4989 init.cc:377] Logged in from keytab as oryx/127.29.233.190@KRBTEST.COM (short username oryx)
I20260810 19:32:39.688743 4989 webserver.cc:460] Webserver started at http://127.29.233.190:45289/ using document root <none> and password file <none>
I20260810 19:32:39.689445 4989 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:39.689594 4989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:39.689908 4989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:39.692952 4989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "cfd2beedb5444b96b423800774185423"
format_stamp: "Formatted at 2026-08-10 19:32:39 on dist-test-slave-tm1g"
server_key: "85a45d5e7084280ee4d303e5397596d7"
server_key_iv: "c446c2944ca8c5fb5e28ae3d539be79d"
server_key_version: "encryptionkey@0"
I20260810 19:32:39.693746 4989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "cfd2beedb5444b96b423800774185423"
format_stamp: "Formatted at 2026-08-10 19:32:39 on dist-test-slave-tm1g"
server_key: "85a45d5e7084280ee4d303e5397596d7"
server_key_iv: "c446c2944ca8c5fb5e28ae3d539be79d"
server_key_version: "encryptionkey@0"
I20260810 19:32:39.699411 4989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 19:32:39.702980 5006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:39.704276 4989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:39.704469 4989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "cfd2beedb5444b96b423800774185423"
format_stamp: "Formatted at 2026-08-10 19:32:39 on dist-test-slave-tm1g"
server_key: "85a45d5e7084280ee4d303e5397596d7"
server_key_iv: "c446c2944ca8c5fb5e28ae3d539be79d"
server_key_version: "encryptionkey@0"
I20260810 19:32:39.704679 4989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:39.741547 4989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:39.747486 4989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:39.747812 4989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:39.766894 4989 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:46487
I20260810 19:32:39.766925 5059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:46487 every 8 connection(s)
I20260810 19:32:39.768337 4989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:39.772138 5060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:39.778215 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 4989
I20260810 19:32:39.778467 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:39.779364 30630 external_mini_cluster.cc:1428] Setting key af8e77745aae0224cef929cf135fbcfd
I20260810 19:32:39.783564 5060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: Bootstrap starting.
I20260810 19:32:39.787212 5060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:39.788259 5060 log.cc:824] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:39.791385 5060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: No bootstrap required, opened a new log
Aug 10 19:32:39 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:39.799981 5060 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } }
I20260810 19:32:39.800357 5060 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:39.800474 5060 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd2beedb5444b96b423800774185423, State: Initialized, Role: FOLLOWER
I20260810 19:32:39.801113 5060 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } }
I20260810 19:32:39.801422 5060 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:39.801578 5060 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:39.801755 5060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:39.804623 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:39.781592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.29.233.190:46487)
0810 19:32:39.782474 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:39.782486 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:39.782527 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:32:39.784442 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:39.784464 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:39.784848 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:39.785195 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:39.787097 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:39.788700 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:39.790268 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:39.791062 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:39.791201 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:39.796094 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:39.796101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:39.796116 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:39.796166 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:39.799741 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:39.800553 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:39.800563 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:39.800571 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:39.800774 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:39.801213 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:39.801217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:39.801221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:39.801993 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:39.802332 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:39.802791 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 19:32:39.803257 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":389,"threads_started":1}
I20260810 19:32:39.805109 5060 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } }
I20260810 19:32:39.805698 5060 leader_election.cc:302] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfd2beedb5444b96b423800774185423; no voters:
I20260810 19:32:39.806519 5060 leader_election.cc:288] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:39.806578 5065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:39.808251 5065 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 1 LEADER]: Becoming Leader. State: Replica: cfd2beedb5444b96b423800774185423, State: Running, Role: LEADER
I20260810 19:32:39.808859 5060 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:39.808796 5065 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } }
I20260810 19:32:39.812055 5070 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd2beedb5444b96b423800774185423. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } }
I20260810 19:32:39.812532 5070 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:39.812484 5069 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } }
I20260810 19:32:39.812947 5069 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:39.815513 5076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:39.820990 5076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:39.831238 5076 catalog_manager.cc:1347] Generated new cluster ID: a9a7add4949e428395feb7252859a6ac
I20260810 19:32:39.831372 5076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:39.848738 5076 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:39.850121 5076 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:39.857244 5076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: Generated new TSK 0
I20260810 19:32:39.858107 5076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab.
Entry for principal oryx/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab.
I20260810 19:32:39.926661 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:46487
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab
--principal=oryx/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:40.269976 5089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:40.270265 5089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:40.270483 5089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:40.277817 5089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:40.277921 5089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:40.278122 5089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:40.287907 5089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab
--principal=oryx/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:46487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:40.289275 5089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:40.290695 5089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:40.303699 5095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:40.303804 5096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:40.303948 5099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:40.304823 5089 server_base.cc:1034] running on GCE node
I20260810 19:32:40.305456 5089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:40.306484 5089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:40.307715 5089 hybrid_clock.cc:648] HybridClock initialized: now 1786390360307673 us; error 33 us; skew 500 ppm
I20260810 19:32:40.308048 5089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:40 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390360, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:40.311714 5089 init.cc:377] Logged in from keytab as oryx/127.29.233.129@KRBTEST.COM (short username oryx)
I20260810 19:32:40.313177 5089 webserver.cc:460] Webserver started at http://127.29.233.129:33359/ using document root <none> and password file <none>
I20260810 19:32:40.313832 5089 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:40.313936 5089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:40.314188 5089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:40.317159 5089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "b03dfb8ab4034d3483dd6db7b65bffe6"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "35ff038f609246c1b1d478c31a8b5301"
server_key_iv: "2cd657778ecd3626366e0678d2b83a25"
server_key_version: "encryptionkey@0"
I20260810 19:32:40.317904 5089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "b03dfb8ab4034d3483dd6db7b65bffe6"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "35ff038f609246c1b1d478c31a8b5301"
server_key_iv: "2cd657778ecd3626366e0678d2b83a25"
server_key_version: "encryptionkey@0"
I20260810 19:32:40.323130 5089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 19:32:40.326678 5106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:40.328016 5089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:32:40.328203 5089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "b03dfb8ab4034d3483dd6db7b65bffe6"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "35ff038f609246c1b1d478c31a8b5301"
server_key_iv: "2cd657778ecd3626366e0678d2b83a25"
server_key_version: "encryptionkey@0"
I20260810 19:32:40.328433 5089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:40.351300 5089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:40.355952 5089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:40.356339 5089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:40.357417 5089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:40.358914 5089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:40.359030 5089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:40.359184 5089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:40.359256 5089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:40.392345 5089 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39067
I20260810 19:32:40.392396 5227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39067 every 8 connection(s)
I20260810 19:32:40.393776 5089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:40.403937 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 5089
I20260810 19:32:40.404191 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:40.404878 30630 external_mini_cluster.cc:1428] Setting key 1fd529a54ab86ceb9bfe52e930a1792b
Aug 10 19:32:40 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390360, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:40.414872 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:40.396647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:42973 (local address 127.29.233.190:46487)
0810 19:32:40.397155 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:40.397160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:40.397985 (+ 825us) server_negotiation.cc:407] Connection header received
0810 19:32:40.399930 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:40.399936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:40.400040 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:40.400163 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:40.401705 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.402528 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.404144 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.404606 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.404747 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:40.408696 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:40.408704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:40.408708 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:40.408751 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:40.411739 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:40.412485 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:40.412492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:40.412495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:40.412630 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:40.413074 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:40.413080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:40.413084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:40.413367 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:40.413537 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:40.414238 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 19:32:40.414499 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":278,"threads_started":1}
I20260810 19:32:40.415876 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:40.397120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:42973)
0810 19:32:40.397816 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:40.397858 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:40.399647 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:40.400421 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:40.400430 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:40.400956 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:40.401524 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:40.401542 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.402702 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.402706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:40.403939 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:40.403951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.404844 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.404852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:40.405142 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:40.405890 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:40.405934 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:40.408454 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:40.411881 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:40.411890 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:40.411908 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:40.412339 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:40.412745 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:40.412750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:40.412754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:40.412954 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:40.413501 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:40.413511 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:40.413859 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 19:32:40.414141 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 19:32:40.414720 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":274,"threads_started":1}
I20260810 19:32:40.417599 5228 heartbeater.cc:347] Connected to a master server at 127.29.233.190:46487
I20260810 19:32:40.417923 5228 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:40.418720 5228 heartbeater.cc:511] Master 127.29.233.190:46487 requested a full tablet report, sending...
I20260810 19:32:40.421157 5023 ts_manager.cc:194] Registered new tserver with Master: b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129:39067)
I20260810 19:32:40.423403 5023 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:42973
WARNING: no policy specified for oryx/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab.
Entry for principal oryx/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab.
I20260810 19:32:40.466459 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:46487
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab
--principal=oryx/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:40.813215 5234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:40.813503 5234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:40.813601 5234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:40.821128 5234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:40.821244 5234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:40.821357 5234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:40.831504 5234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab
--principal=oryx/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:46487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:40.832901 5234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:40.834342 5234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:40.847992 5239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:40.848011 5240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:40.848289 5242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:40.848733 5234 server_base.cc:1034] running on GCE node
I20260810 19:32:40.849370 5234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:40.850391 5234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:40.851657 5234 hybrid_clock.cc:648] HybridClock initialized: now 1786390360851616 us; error 45 us; skew 500 ppm
I20260810 19:32:40.851985 5234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:40 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390360, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:40.855160 5234 init.cc:377] Logged in from keytab as oryx/127.29.233.130@KRBTEST.COM (short username oryx)
I20260810 19:32:40.856722 5234 webserver.cc:460] Webserver started at http://127.29.233.130:36525/ using document root <none> and password file <none>
I20260810 19:32:40.857488 5234 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:40.857640 5234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:40.857963 5234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:40.860999 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "7006b47b0979434d9683a6f340627ee4"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "b1b7bde15bf6e00bf0c9a19332fb0292"
server_key_iv: "d3277acf20ed50cf8bd59c5d1d74e170"
server_key_version: "encryptionkey@0"
I20260810 19:32:40.861892 5234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "7006b47b0979434d9683a6f340627ee4"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "b1b7bde15bf6e00bf0c9a19332fb0292"
server_key_iv: "d3277acf20ed50cf8bd59c5d1d74e170"
server_key_version: "encryptionkey@0"
I20260810 19:32:40.867230 5234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 19:32:40.870784 5249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:40.872233 5234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:32:40.872414 5234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "7006b47b0979434d9683a6f340627ee4"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "b1b7bde15bf6e00bf0c9a19332fb0292"
server_key_iv: "d3277acf20ed50cf8bd59c5d1d74e170"
server_key_version: "encryptionkey@0"
I20260810 19:32:40.872632 5234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:40.885085 5234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:40.889070 5234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:40.889365 5234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:40.890581 5234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:40.891995 5234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:40.892103 5234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:40.892254 5234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:40.892329 5234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:40.927956 5234 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:42911
I20260810 19:32:40.928018 5369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:42911 every 8 connection(s)
I20260810 19:32:40.929271 5234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:40.932879 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 5234
I20260810 19:32:40.933120 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:40.933758 30630 external_mini_cluster.cc:1428] Setting key 9b9d97cb71dcca21dae38bb918d128b8
Aug 10 19:32:40 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390360, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:40.952440 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:40.931926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:60581 (local address 127.29.233.190:46487)
0810 19:32:40.932455 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:40.932460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:40.933701 (+ 1241us) server_negotiation.cc:407] Connection header received
0810 19:32:40.935987 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:40.935993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:40.936076 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:40.936202 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:40.937817 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.939010 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.940603 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.941322 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.941469 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:40.945219 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:40.945229 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:40.945236 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:40.945284 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:40.948870 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:40.949609 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:40.949617 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:40.949624 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:40.949802 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:40.950236 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:40.950244 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:40.950250 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:40.950750 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:40.950911 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:40.951613 (+ 702us) server_negotiation.cc:299] Negotiation successful
0810 19:32:40.951876 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":326,"threads_started":1}
I20260810 19:32:40.953538 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:40.932556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.130:60581)
0810 19:32:40.933543 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:40.933589 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:40.935596 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:40.936489 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:40.936499 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:40.937031 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:40.937617 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:40.937634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.939164 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.939169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:40.940447 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:40.940460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:40.941535 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:40.941540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:40.941731 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:40.942523 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:40.942548 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:40.944960 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:40.949018 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:40.949025 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:40.949040 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:40.949484 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:40.949915 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:40.949918 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:40.949921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:40.950117 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:40.950881 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:40.950888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:40.951230 (+ 342us) client_negotiation.cc:769] Sending connection context
0810 19:32:40.951472 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 19:32:40.951898 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":368,"threads_started":1}
I20260810 19:32:40.955155 5370 heartbeater.cc:347] Connected to a master server at 127.29.233.190:46487
I20260810 19:32:40.955559 5370 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:40.956463 5370 heartbeater.cc:511] Master 127.29.233.190:46487 requested a full tablet report, sending...
I20260810 19:32:40.958214 5023 ts_manager.cc:194] Registered new tserver with Master: 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911)
I20260810 19:32:40.959196 5023 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:60581
WARNING: no policy specified for oryx/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab.
Entry for principal oryx/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab.
I20260810 19:32:40.997298 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:46487
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab
--principal=oryx/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:41.337718 5376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:41.338075 5376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:41.338253 5376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:41.345728 5376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:41.345888 5376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:41.346084 5376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:41.357299 5376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab
--principal=oryx/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:46487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:41.358788 5376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:41.360181 5376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:41.374158 5384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:41.374189 5382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:41.374723 5381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:41.374876 5376 server_base.cc:1034] running on GCE node
I20260810 19:32:41.375519 5376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:41.376523 5376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:41.377748 5376 hybrid_clock.cc:648] HybridClock initialized: now 1786390361377707 us; error 51 us; skew 500 ppm
I20260810 19:32:41.378082 5376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:41 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390361, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:41.388280 5376 init.cc:377] Logged in from keytab as oryx/127.29.233.131@KRBTEST.COM (short username oryx)
I20260810 19:32:41.389781 5376 webserver.cc:460] Webserver started at http://127.29.233.131:33551/ using document root <none> and password file <none>
I20260810 19:32:41.390604 5376 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:41.390763 5376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:41.391108 5376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:41.394184 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "f7f2920c516d4db2b3a85bda54bbbd8c"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "31dca7a38fa9574a3b3cf9c7c7d9125a"
server_key_iv: "29fb7a3d981bbec3de32c72cd27367c7"
server_key_version: "encryptionkey@0"
I20260810 19:32:41.395066 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "f7f2920c516d4db2b3a85bda54bbbd8c"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "31dca7a38fa9574a3b3cf9c7c7d9125a"
server_key_iv: "29fb7a3d981bbec3de32c72cd27367c7"
server_key_version: "encryptionkey@0"
I20260810 19:32:41.400429 5376 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:32:41.403895 5391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:41.405207 5376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:32:41.405396 5376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "f7f2920c516d4db2b3a85bda54bbbd8c"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "31dca7a38fa9574a3b3cf9c7c7d9125a"
server_key_iv: "29fb7a3d981bbec3de32c72cd27367c7"
server_key_version: "encryptionkey@0"
I20260810 19:32:41.405601 5376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:41.426616 5228 heartbeater.cc:503] Master 127.29.233.190:46487 was elected leader, sending a full tablet report...
I20260810 19:32:41.441222 5376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:41.445833 5376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:41.446149 5376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:41.447237 5376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:41.448669 5376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:41.448778 5376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:41.448944 5376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:41.449020 5376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:41.484098 5376 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:45957
I20260810 19:32:41.484145 5512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:45957 every 8 connection(s)
I20260810 19:32:41.485749 5376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:41.495049 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 5376
I20260810 19:32:41.495253 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:41.495733 30630 external_mini_cluster.cc:1428] Setting key 1bf68d89a5837d601116d3ededf33870
Aug 10 19:32:41 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390361, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:41.507053 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:41.488505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:37951 (local address 127.29.233.190:46487)
0810 19:32:41.489042 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:41.489047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:41.489856 (+ 809us) server_negotiation.cc:407] Connection header received
0810 19:32:41.491756 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:41.491763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:41.491878 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:41.492003 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:41.493556 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:41.494470 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:41.496239 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:41.496820 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:41.496970 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:41.500813 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:41.500819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:41.500822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:41.500862 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:41.503840 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:41.504601 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:41.504608 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:41.504614 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:41.504750 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:41.505198 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:41.505204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:41.505207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:41.505510 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:41.505680 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:41.506426 (+ 746us) server_negotiation.cc:299] Negotiation successful
0810 19:32:41.506652 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":292,"threads_started":1}
I20260810 19:32:41.508106 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:41.488955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.131:37951)
0810 19:32:41.489681 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:41.489720 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:41.491472 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:41.492256 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:41.492265 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:41.492791 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:41.493372 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:41.493390 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:41.494634 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:41.494638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:41.496082 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:41.496095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:41.497067 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:41.497077 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:41.497335 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:41.498061 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:41.498089 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:41.500589 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:41.503985 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:41.503995 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:41.504013 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:41.504471 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:41.504875 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:41.504880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:41.504884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:41.505078 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:41.505647 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:41.505655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:41.506027 (+ 372us) client_negotiation.cc:769] Sending connection context
0810 19:32:41.506350 (+ 323us) client_negotiation.cc:240] Negotiation successful
0810 19:32:41.506928 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":329,"threads_started":1}
I20260810 19:32:41.509905 5513 heartbeater.cc:347] Connected to a master server at 127.29.233.190:46487
I20260810 19:32:41.510263 5513 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:41.511350 5513 heartbeater.cc:511] Master 127.29.233.190:46487 requested a full tablet report, sending...
I20260810 19:32:41.512941 5023 ts_manager.cc:194] Registered new tserver with Master: f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957)
I20260810 19:32:41.513814 5023 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:37951
I20260810 19:32:41.524235 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.233.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.189.keytab.
Entry for principal oryx/127.29.233.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.189.keytab.
I20260810 19:32:41.586519 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:34267
--webserver_interface=127.29.233.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.189.keytab
--principal=oryx/127.29.233.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46487,127.29.233.189:34267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:41.937870 5519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:41.938160 5519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:41.938381 5519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:41.945892 5519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:41.946002 5519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:41.946094 5519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:41.946174 5519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:41.946244 5519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:41.956598 5519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46487,127.29.233.189:34267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.189.keytab
--principal=oryx/127.29.233.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.233.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
W20260810 19:32:41.957007 5519 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.233.190:46487,127.29.233.189:34267'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 19:32:41.958159 5519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:41.959546 5519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 19:32:41.962596 5370 heartbeater.cc:503] Master 127.29.233.190:46487 was elected leader, sending a full tablet report...
W20260810 19:32:41.971974 5524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:41.971920 5525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:41.971952 5527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:41.975780 5519 server_base.cc:1034] running on GCE node
I20260810 19:32:41.976621 5519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:41.978120 5519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:41.979372 5519 hybrid_clock.cc:648] HybridClock initialized: now 1786390361979294 us; error 49 us; skew 500 ppm
I20260810 19:32:41.979722 5519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:41 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390361, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:41.983052 5519 init.cc:377] Logged in from keytab as oryx/127.29.233.189@KRBTEST.COM (short username oryx)
I20260810 19:32:41.984555 5519 webserver.cc:460] Webserver started at http://127.29.233.189:42801/ using document root <none> and password file <none>
I20260810 19:32:41.985253 5519 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:41.985357 5519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:41.985642 5519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:41.988739 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/instance:
uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "29ca84dc35c05e8b41f55a20b1480283"
server_key_iv: "8f216aca223617e0acbd31c51eecdf74"
server_key_version: "encryptionkey@0"
I20260810 19:32:41.989467 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal/instance:
uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "29ca84dc35c05e8b41f55a20b1480283"
server_key_iv: "8f216aca223617e0acbd31c51eecdf74"
server_key_version: "encryptionkey@0"
I20260810 19:32:41.994783 5519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 19:32:41.998108 5535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:41.999495 5519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:41.999644 5519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "29ca84dc35c05e8b41f55a20b1480283"
server_key_iv: "8f216aca223617e0acbd31c51eecdf74"
server_key_version: "encryptionkey@0"
I20260810 19:32:41.999949 5519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:42.020052 5519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:42.024464 5519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:42.024753 5519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:42.044689 5519 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.189:34267
I20260810 19:32:42.044734 5592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.189:34267 every 8 connection(s)
I20260810 19:32:42.046204 5519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
I20260810 19:32:42.050843 5593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:42.053557 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 5519
I20260810 19:32:42.058415 5593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.189@KRBTEST.COM
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390361, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.189@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:42.077359 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.061444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59636 (local address 127.29.233.190:46487)
0810 19:32:42.061961 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.061970 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.062531 (+ 561us) server_negotiation.cc:407] Connection header received
0810 19:32:42.062772 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.062777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.062852 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.062969 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.064076 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.064833 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.066525 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.066926 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.067069 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.070815 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.070820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.070823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.070862 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.074159 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.075035 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.075039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.075043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.075205 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.075682 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.075687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.075690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.076110 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.076272 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.076779 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.077031 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":270,"threads_started":1}
I20260810 19:32:42.077523 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.055967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.29.233.189:34267)
0810 19:32:42.056815 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.056827 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.056866 (+ 39us) server_negotiation.cc:407] Connection header received
0810 19:32:42.058656 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.058679 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.059082 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.059434 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.061359 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.063193 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.064469 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.065115 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.065365 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.069759 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.069771 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.069787 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.069829 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.072659 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.073524 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.073530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.073537 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.073685 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.074114 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.074117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.074119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.074894 (+ 775us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.075164 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.075672 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.076122 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":629,"thread_start_us":420,"threads_started":1}
I20260810 19:32:42.078140 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.061543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:59636)
0810 19:32:42.062353 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.062427 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.062616 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.063041 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.063048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.063665 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.063913 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.063922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.064984 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.064990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.066366 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.066386 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.067176 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.067184 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.067344 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.067944 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.067965 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.070583 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.074273 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.074281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.074289 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.074892 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.075319 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.075322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.075324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.075527 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.076255 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.076261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.076402 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.076558 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.076915 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":420,"threads_started":1}
I20260810 19:32:42.079882 5593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390361, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.189@KRBTEST.COM for oryx/127.29.233.189@KRBTEST.COM
I20260810 19:32:42.091653 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.080555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.0.0.1:42518)
0810 19:32:42.080764 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.080780 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.080857 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.081097 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.081100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.081401 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.081559 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.081567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.082568 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.082572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.083395 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.083414 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.084052 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.084056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.084189 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.084688 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.084704 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.086741 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.089616 (+ 2875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.089619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.089621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.090021 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.090411 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.090416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.090419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.090544 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.091010 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.091013 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.091115 (+ 102us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.091229 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.091421 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 19:32:42.091869 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.080733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42518 (local address 127.29.233.189:34267)
0810 19:32:42.080857 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.080860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.080875 (+ 15us) server_negotiation.cc:407] Connection header received
0810 19:32:42.080932 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.080936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.080992 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.081064 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.081675 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.082435 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.083532 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.083921 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.084025 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.086938 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.086946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.086950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.086981 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.089491 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.090121 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.090124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.090126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.090241 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.090645 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.090648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.090650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.090913 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.091027 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.091370 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.091531 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 19:32:42.096745 5593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Bootstrap starting.
I20260810 19:32:42.099876 5593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:42.100951 5593 log.cc:824] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:42.103826 5593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: No bootstrap required, opened a new log
I20260810 19:32:42.111541 5593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } }
I20260810 19:32:42.111855 5593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:42.111929 5593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e35c1c720704f8e9fd1433d2bfc6fb1, State: Initialized, Role: FOLLOWER
I20260810 19:32:42.112499 5593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } }
I20260810 19:32:42.113304 5600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } } }
I20260810 19:32:42.113652 5600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:32:42.114404 5593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 19:32:42.124967 5612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 19:32:42.125123 5612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 19:32:42.127118 5519 master_runner.cc:428] Detected that this master 7e35c1c720704f8e9fd1433d2bfc6fb1 is joining an existing cluster
I20260810 19:32:42.127233 5519 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.233.189:34267
I20260810 19:32:42.151101 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.131181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.29.233.190:46487)
0810 19:32:42.131376 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.131383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.134984 (+ 3601us) server_negotiation.cc:407] Connection header received
0810 19:32:42.135272 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.135278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.135385 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.135462 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.137114 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.138240 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.140497 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.141247 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.141370 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.144232 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.144238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.144243 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.144282 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.147725 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.148544 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.148550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.148554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.148725 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.149184 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.149189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.149193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.149592 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.149746 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.150379 (+ 633us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.150660 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 19:32:42.152246 5618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.131146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:59638)
0810 19:32:42.134874 (+ 3728us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.134901 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.135107 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.135708 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.135715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.136446 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.136923 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.136939 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.138458 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.138466 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.140320 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.140339 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.141688 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.141696 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.141923 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.142678 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.142711 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.144023 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.147879 (+ 3856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.147890 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.147896 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.148394 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.148867 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.148875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.148881 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.149061 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.149820 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.149824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.150027 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.151373 (+ 1346us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.151742 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3525,"thread_start_us":3370,"threads_started":1}
I20260810 19:32:42.160030 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.135223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.29.233.189:34267)
0810 19:32:42.135422 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.135429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.143987 (+ 8558us) server_negotiation.cc:407] Connection header received
0810 19:32:42.144203 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.144214 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.144280 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.144354 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.145835 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.146978 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.148959 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.149709 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.149800 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.152632 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.152640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.152644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.152680 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.156885 (+ 4205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.157772 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.157777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.157780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.157921 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.158460 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.158465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.158468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.158812 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.158943 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.159438 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.159672 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for oryx/127.29.233.188@KRBTEST.COM; defaulting to no policy
I20260810 19:32:42.167147 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.135382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.0.0.1:42532)
0810 19:32:42.143890 (+ 8508us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.143912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.144082 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.144532 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.144536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.145277 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.145663 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.145677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.147191 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.147198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.148823 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.148835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.150125 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.150131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.150344 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.150943 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.150965 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.152281 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.157072 (+ 4791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.157077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.157081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.157657 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.158075 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.158079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.158081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.158286 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.159033 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.159037 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.159171 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.166556 (+ 7385us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.166813 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8350,"thread_start_us":8116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.188@KRBTEST.COM" created.
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.190@KRBTEST.COM for oryx/127.29.233.189@KRBTEST.COM
I20260810 19:32:42.184206 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.160254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.29.233.189:34267)
0810 19:32:42.162064 (+ 1810us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.162073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.166023 (+ 3950us) server_negotiation.cc:407] Connection header received
0810 19:32:42.166507 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.166515 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.166651 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.166732 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.168868 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.169993 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.172025 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.172784 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.172927 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.177341 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.177349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.177354 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.177395 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.181165 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.181979 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.181984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.181986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.182137 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.182631 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.182635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.182637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.182924 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.183106 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.183573 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.183801 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1662,"thread_start_us":6194,"threads_started":1}
I20260810 19:32:42.184257 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.159871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.0.0.1:42544)
0810 19:32:42.165868 (+ 5997us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.165927 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.166209 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.167022 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.167031 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.167916 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.168690 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.168704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.170174 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.170180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.171837 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.171859 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.172979 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.172987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.173193 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.173972 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.173998 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.177143 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.181304 (+ 4161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.181315 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.181321 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.181831 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.182255 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.182260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.182265 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.182490 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.183066 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.183075 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.183272 (+ 197us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.183470 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.183803 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5717,"thread_start_us":5548,"threads_started":1}
I20260810 19:32:42.186566 5023 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.233.189:34267: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1"
member_type: NON_VOTER
last_known_addr {
host: "127.29.233.189"
port: 34267
}
attrs {
promote: true
}
}
dest_uuid: "cfd2beedb5444b96b423800774185423"
cas_config_opid_index: -1
I20260810 19:32:42.187829 5023 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } }
I20260810 19:32:42.196033 5625 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7e35c1c720704f8e9fd1433d2bfc6fb1 (127.29.233.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } }
I20260810 19:32:42.197119 5564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:42.200383 5625 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.233.189:34267
I20260810 19:32:42.201572 5519 master.cc:537] Master@127.29.233.189:34267 shutting down...
I20260810 19:32:42.201756 5564 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Refusing update from remote peer cfd2beedb5444b96b423800774185423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 19:32:42.200986 5626 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } }
I20260810 19:32:42.202167 5626 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:42.202859 5625 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 19:32:42.206593 5010 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 19:32:42.209542 5010 consensus_peers.cc:487] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 -> Peer 7e35c1c720704f8e9fd1433d2bfc6fb1 (127.29.233.189:34267): Couldn't send request to peer 7e35c1c720704f8e9fd1433d2bfc6fb1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.188.keytab.
Entry for principal oryx/127.29.233.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.188.keytab.
I20260810 19:32:42.214355 5519 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:42.214715 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:40533
--webserver_interface=127.29.233.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.188.keytab
--principal=oryx/127.29.233.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:32:42.214872 5519 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:42.215085 5519 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: stopping tablet replica
I20260810 19:32:42.223307 5519 master.cc:552] Master@127.29.233.189:34267 shutdown complete.
I20260810 19:32:42.223459 5519 master_runner.cc:305] Clearing existing system tablet
I20260810 19:32:42.223970 5519 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:42.224076 5519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:42.228036 5519 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:32:42.229601 5631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:42.229914 5519 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:42.230224 5519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 19:32:42.230337 5519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "29ca84dc35c05e8b41f55a20b1480283"
server_key_iv: "8f216aca223617e0acbd31c51eecdf74"
server_key_version: "encryptionkey@0"
I20260810 19:32:42.234097 5519 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:42.239048 5519 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 19:32:42.239252 5519 log.cc:1196] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 19:32:42.239759 5519 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Deleting consensus metadata
I20260810 19:32:42.240052 5519 master_runner.cc:315] Copying system tablet from 127.29.233.190:46487
I20260810 19:32:42.246192 5519 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: tablet copy: Beginning tablet copy session from remote peer at address 127.29.233.190:46487
I20260810 19:32:42.266690 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.247024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.29.233.190:46487)
0810 19:32:42.247249 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.247258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.255299 (+ 8041us) server_negotiation.cc:407] Connection header received
0810 19:32:42.255534 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.255540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.255621 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.255734 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.256631 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.257393 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.258689 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.259110 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.259244 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.261165 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.261172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.261176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.261218 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.263986 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.264650 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.264657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.264661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.264790 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.265167 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.265172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.265176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.265469 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.265634 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.266034 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.266252 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:32:42.266822 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.247012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:59652)
0810 19:32:42.255168 (+ 8156us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.255190 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.255376 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.255769 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.255774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.256233 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.256468 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.256478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.257548 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.257555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.258551 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.258564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.259261 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.259267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.259466 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.260218 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.260236 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.260967 (+ 731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.264139 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.264144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.264153 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.264535 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.264903 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.264908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.264916 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.265053 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.265595 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.265600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.265774 (+ 174us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.265936 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.266257 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7999,"thread_start_us":7896,"threads_started":1}
I20260810 19:32:42.267879 5044 tablet_copy_service.cc:140] P cfd2beedb5444b96b423800774185423: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7e35c1c720704f8e9fd1433d2bfc6fb1 ({username='oryx', principal='oryx/127.29.233.189@KRBTEST.COM'} at 127.0.0.1:59652)
I20260810 19:32:42.268170 5044 tablet_copy_service.cc:161] P cfd2beedb5444b96b423800774185423: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7e35c1c720704f8e9fd1433d2bfc6fb1 at {username='oryx', principal='oryx/127.29.233.189@KRBTEST.COM'} at 127.0.0.1:59652: session id = 7e35c1c720704f8e9fd1433d2bfc6fb1-00000000000000000000000000000000
I20260810 19:32:42.271314 5044 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 19:32:42.273187 5519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 19:32:42.278242 5519 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: tablet copy: Starting download of 0 data blocks...
I20260810 19:32:42.278595 5519 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: tablet copy: Starting download of 1 WAL segments...
I20260810 19:32:42.281085 5519 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 19:32:42.284411 5044 tablet_copy_service.cc:342] P cfd2beedb5444b96b423800774185423: Request end of tablet copy session 7e35c1c720704f8e9fd1433d2bfc6fb1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.233.189@KRBTEST.COM'} at 127.0.0.1:59652
I20260810 19:32:42.284621 5044 tablet_copy_service.cc:433] P cfd2beedb5444b96b423800774185423: ending tablet copy session 7e35c1c720704f8e9fd1433d2bfc6fb1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7e35c1c720704f8e9fd1433d2bfc6fb1
I20260810 19:32:42.295104 5519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:42.297799 5639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:42.308226 5642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:42.310559 5640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:42.311151 5519 server_base.cc:1034] running on GCE node
I20260810 19:32:42.311534 5519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:42.315972 5519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:42.317238 5519 hybrid_clock.cc:648] HybridClock initialized: now 1786390362317152 us; error 57 us; skew 500 ppm
I20260810 19:32:42.317489 5519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390362, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:42.319782 5519 init.cc:377] Logged in from keytab as oryx/127.29.233.189@KRBTEST.COM (short username oryx)
I20260810 19:32:42.321214 5519 webserver.cc:460] Webserver started at http://127.29.233.189:35915/ using document root <none> and password file <none>
I20260810 19:32:42.321611 5519 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:42.321696 5519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:42.324968 5519 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 19:32:42.327142 5652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:42.327620 5519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 19:32:42.327764 5519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "29ca84dc35c05e8b41f55a20b1480283"
server_key_iv: "8f216aca223617e0acbd31c51eecdf74"
server_key_version: "encryptionkey@0"
I20260810 19:32:42.327934 5519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:42.354979 5519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:42.359151 5519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:42.405375 5519 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.189:34267
I20260810 19:32:42.405434 5703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.189:34267 every 8 connection(s)
I20260810 19:32:42.406468 5519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
I20260810 19:32:42.411679 5704 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:32:42.416124 5704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Bootstrap starting.
I20260810 19:32:42.435889 5704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:32:42.436450 5704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Bootstrap complete.
I20260810 19:32:42.437548 5704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } }
I20260810 19:32:42.437839 5704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e35c1c720704f8e9fd1433d2bfc6fb1, State: Initialized, Role: LEARNER
I20260810 19:32:42.438081 5704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } }
I20260810 19:32:42.438931 5707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } }
I20260810 19:32:42.439203 5704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:42.439343 5707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: LEARNER
I20260810 19:32:42.450104 5718 catalog_manager.cc:1259] Loaded cluster ID: a9a7add4949e428395feb7252859a6ac
I20260810 19:32:42.450222 5718 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: loading cluster ID for follower catalog manager: success
I20260810 19:32:42.453689 5718 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: acquiring CA information for follower catalog manager: success
I20260810 19:32:42.456032 5718 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 19:32:42.516561 5513 heartbeater.cc:503] Master 127.29.233.190:46487 was elected leader, sending a full tablet report...
W20260810 19:32:42.619424 5629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:42.619824 5629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:42.620002 5629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:42.627657 5629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:42.627810 5629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:42.627921 5629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:42.628036 5629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:42.628113 5629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:42.638772 5629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.188.keytab
--principal=oryx/127.29.233.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:40533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.233.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:42.640393 5629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:42.641788 5629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:42.654016 5728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:42.654047 5724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:42.654078 5725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:42.655733 5629 server_base.cc:1034] running on GCE node
I20260810 19:32:42.656605 5629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:42.658092 5629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:42.659372 5629 hybrid_clock.cc:648] HybridClock initialized: now 1786390362659308 us; error 58 us; skew 500 ppm
I20260810 19:32:42.659792 5629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390362, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:42.663185 5629 init.cc:377] Logged in from keytab as oryx/127.29.233.188@KRBTEST.COM (short username oryx)
I20260810 19:32:42.664808 5629 webserver.cc:460] Webserver started at http://127.29.233.188:35475/ using document root <none> and password file <none>
I20260810 19:32:42.665558 5629 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:42.665709 5629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:42.666035 5629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:42.669179 5629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/instance:
uuid: "519617e664494d928d11e37fb078ba8a"
format_stamp: "Formatted at 2026-08-10 19:32:42 on dist-test-slave-tm1g"
server_key: "235dfac3907c4df337e94a3952e494cf"
server_key_iv: "ebe8766982cfa1a5b5ebc05e75fabddd"
server_key_version: "encryptionkey@0"
I20260810 19:32:42.669983 5629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal/instance:
uuid: "519617e664494d928d11e37fb078ba8a"
format_stamp: "Formatted at 2026-08-10 19:32:42 on dist-test-slave-tm1g"
server_key: "235dfac3907c4df337e94a3952e494cf"
server_key_iv: "ebe8766982cfa1a5b5ebc05e75fabddd"
server_key_version: "encryptionkey@0"
I20260810 19:32:42.675544 5629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 19:32:42.679597 5736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:42.681123 5629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260810 19:32:42.681275 5629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
uuid: "519617e664494d928d11e37fb078ba8a"
format_stamp: "Formatted at 2026-08-10 19:32:42 on dist-test-slave-tm1g"
server_key: "235dfac3907c4df337e94a3952e494cf"
server_key_iv: "ebe8766982cfa1a5b5ebc05e75fabddd"
server_key_version: "encryptionkey@0"
I20260810 19:32:42.681457 5629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:42.700196 5629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:42.704885 5629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:42.705129 5629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:42.725102 5629 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.188:40533
I20260810 19:32:42.725147 5793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.188:40533 every 8 connection(s)
I20260810 19:32:42.726727 5629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
I20260810 19:32:42.730800 5794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:42.730993 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 5629
I20260810 19:32:42.741411 5794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.188@KRBTEST.COM
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390362, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.188@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:42.756604 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.733095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.29.233.188:40533)
0810 19:32:42.733996 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.734011 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.734055 (+ 44us) server_negotiation.cc:407] Connection header received
0810 19:32:42.735865 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.735888 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.736273 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.736626 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.738637 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.740105 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.742134 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.743065 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.743199 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.748531 (+ 5332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.748540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.748551 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.748590 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.752100 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.752837 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.752853 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.752860 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.753059 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.753479 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.753484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.753488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.754014 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.754352 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.754853 (+ 501us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.755482 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":442,"threads_started":1}
I20260810 19:32:42.759652 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.744721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.29.233.190:46487)
0810 19:32:42.744989 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.744998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.745720 (+ 722us) server_negotiation.cc:407] Connection header received
0810 19:32:42.745923 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.745928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.746001 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.746084 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.747558 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.748256 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.749772 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.750185 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.750366 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.753789 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.753793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.753797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.753831 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.756612 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.757480 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.757485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.757489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.757664 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.758054 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.758059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.758064 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.758421 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.758584 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.759050 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.759291 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 19:32:42.759727 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.744654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:59668)
0810 19:32:42.745555 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.745607 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.745799 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.746195 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.746202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.747101 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.747366 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.747375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.748422 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.748432 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.749590 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.749604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.750402 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.750409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.750606 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.751253 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.751278 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.753592 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.756776 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.756788 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.756794 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.757333 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.757781 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.757784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.757786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.757931 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.758557 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.758565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.758736 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.758953 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.759291 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":430,"threads_started":1}
I20260810 19:32:42.761175 5794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390362, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.188@KRBTEST.COM for oryx/127.29.233.189@KRBTEST.COM
I20260810 19:32:42.776065 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.761828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.0.0.1:42560)
0810 19:32:42.762058 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.762076 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.762177 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.762978 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.762982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.763298 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.763435 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.763443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.764707 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.764711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.765684 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.765696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.766417 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.766424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.766592 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.767287 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.767316 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.769620 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.773591 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.773595 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.773597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.774004 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.774461 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.774464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.774466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.774604 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.775193 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.775196 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.775320 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.775446 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.775831 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 19:32:42.776171 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.762081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.29.233.189:34267)
0810 19:32:42.762718 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.762722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.762738 (+ 16us) server_negotiation.cc:407] Connection header received
0810 19:32:42.762786 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.762790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.762847 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.762944 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.763590 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.764590 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.765810 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.766222 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.766374 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.769812 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.769819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.769824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.769863 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.773439 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.774117 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.774123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.774127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.774286 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.774709 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.774715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.774718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.775063 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.775216 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.775571 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.775791 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":379,"threads_started":1}
I20260810 19:32:42.777264 5794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390362, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.188@KRBTEST.COM for oryx/127.29.233.188@KRBTEST.COM
I20260810 19:32:42.791184 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.778289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.0.0.1:34688)
0810 19:32:42.778518 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.778537 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.778646 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.778929 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.778935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.779382 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.779552 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.779562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.780522 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.780527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.781909 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.781923 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.782672 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.782678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.782903 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.783469 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.783489 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.786004 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.788780 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.788784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.788787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.789153 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.789508 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.789512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.789514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.789724 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.790380 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.790387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.790534 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.790663 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.790880 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 19:32:42.791333 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.778161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.29.233.188:40533)
0810 19:32:42.778452 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.778459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.778635 (+ 176us) server_negotiation.cc:407] Connection header received
0810 19:32:42.778738 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.778742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.778797 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.778873 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.779698 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.780376 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.782038 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.782510 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.782609 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.786176 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.786182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.786186 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.786220 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.788642 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.789256 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.789260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.789262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.789367 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.789848 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.789853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.789856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.790185 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.790348 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.790754 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.791039 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":70}
I20260810 19:32:42.796367 5794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Bootstrap starting.
I20260810 19:32:42.799635 5794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:42.800691 5794 log.cc:824] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:42.803699 5794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: No bootstrap required, opened a new log
I20260810 19:32:42.812758 5794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } }
I20260810 19:32:42.813134 5794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:42.813289 5794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519617e664494d928d11e37fb078ba8a, State: Initialized, Role: FOLLOWER
I20260810 19:32:42.813750 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.806908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.29.233.189:34267)
0810 19:32:42.807140 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.807148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.807359 (+ 211us) server_negotiation.cc:407] Connection header received
0810 19:32:42.807578 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.807594 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.807697 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.807822 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:42.808267 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.809205 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.811655 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.812591 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.812800 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.813033 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.813325 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.813491 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 19:32:42.813844 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.806638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.0.0.1:42568)
0810 19:32:42.807205 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.807223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.807409 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.807887 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.807892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.807931 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:42.808114 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.808123 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.809430 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.809435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.811474 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.811489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.812796 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.812803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.813071 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.813137 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.813245 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.813408 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":278,"threads_started":1}
I20260810 19:32:42.814013 5794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } }
I20260810 19:32:42.815045 5679 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 19:32:42.815083 5806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } } }
I20260810 19:32:42.815471 5806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:32:42.815652 5707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd2beedb5444b96b423800774185423. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } }
I20260810 19:32:42.815968 5707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: LEARNER
I20260810 19:32:42.816531 5794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: configured and running, proceeding with master startup.
W20260810 19:32:42.826224 5818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 19:32:42.826478 5818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 19:32:42.830544 5629 master_runner.cc:428] Detected that this master 519617e664494d928d11e37fb078ba8a is joining an existing cluster
I20260810 19:32:42.830655 5629 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.233.188:40533
I20260810 19:32:42.855278 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.834948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.29.233.190:46487)
0810 19:32:42.835346 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.835354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.836362 (+ 1008us) server_negotiation.cc:407] Connection header received
0810 19:32:42.836590 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.836597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.836685 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.836796 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.838471 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.839737 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.841978 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.842664 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.842768 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.846287 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.846293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.846327 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.846386 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.851401 (+ 5015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.852505 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.852512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.852517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.852688 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.853218 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.853223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.853227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.853663 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.853792 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.854610 (+ 818us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.854863 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258}
I20260810 19:32:42.855777 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.835140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.29.233.188:40533)
0810 19:32:42.835563 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.835572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.837158 (+ 1586us) server_negotiation.cc:407] Connection header received
0810 19:32:42.837415 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.837424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.837497 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.837573 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.839458 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.841011 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.844183 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.845097 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.845236 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.848376 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.848383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.848388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.848433 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.852656 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.853520 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.853525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.853528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.853691 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.854147 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.854152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.854155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.854531 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.854685 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.855180 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.855401 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260810 19:32:42.856858 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.835250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.0.0.1:34698)
0810 19:32:42.837065 (+ 1815us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.837083 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.837242 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.837763 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.837768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.838804 (+ 1036us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.839257 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.839273 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.841282 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.841293 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.844016 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.844030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.846034 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.846043 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.846220 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.846960 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.846984 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.848178 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.852804 (+ 4626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.852809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.852817 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.853375 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.853847 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.853851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.853855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.854023 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.854752 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.854757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.854922 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.856043 (+ 1121us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.856373 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1669,"mutex_wait_us":520,"thread_start_us":587,"threads_started":1}
I20260810 19:32:42.857540 5825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.835063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:59674)
0810 19:32:42.836217 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.836242 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.836436 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.837005 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.837010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.837777 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.838239 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.838257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.839972 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.839978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.841799 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.841818 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.843084 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.843091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.843400 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.844212 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.844239 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.845814 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.851580 (+ 5766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.851588 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.851593 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.852346 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.852850 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.852855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.852863 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.853081 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.853899 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.853904 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.854151 (+ 247us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.856142 (+ 1991us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.856564 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":97,"thread_start_us":1672,"threads_started":1}
I20260810 19:32:42.860304 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.837571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.29.233.189:34267)
0810 19:32:42.837760 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.837766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.843892 (+ 6126us) server_negotiation.cc:407] Connection header received
0810 19:32:42.844204 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.844210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.844300 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.844376 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.846138 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.847304 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.849571 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.850255 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.850392 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.853404 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.853411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.853415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.853455 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.856995 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.857851 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.857858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.857861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.858046 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.858549 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.858554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.858557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.858897 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.859025 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.859686 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.859943 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:32:42.860357 5826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.837178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.0.0.1:42574)
0810 19:32:42.843773 (+ 6595us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.843795 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.844011 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.844618 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.844624 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.845548 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.845942 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.845955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.847517 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.847522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.849390 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.849406 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.850756 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.850764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.850963 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.851601 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.851692 (+ 91us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.853202 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.857170 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.857175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.857180 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.857706 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.858205 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.858209 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.858212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.858413 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.859125 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.859129 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.859372 (+ 243us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.859552 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.859903 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6390,"spinlock_wait_cycles":28928,"thread_start_us":6261,"threads_started":1}
I20260810 19:32:42.863044 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 5089
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.190@KRBTEST.COM for oryx/127.29.233.188@KRBTEST.COM
Aug 10 19:32:42 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.129@KRBTEST.COM
I20260810 19:32:42.880527 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.863934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.0.0.1:34702)
0810 19:32:42.864139 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.864158 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.864271 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.864858 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.864863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.865388 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.865594 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.865608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.867280 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.867286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.868728 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.868740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.869497 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.869503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.869694 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.870291 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.870356 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.872848 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:42.877155 (+ 4307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.877161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.877165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.877729 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.878167 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:42.878175 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:42.878180 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:42.878414 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:42.879179 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:42.879184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:42.879701 (+ 517us) client_negotiation.cc:769] Sending connection context
0810 19:32:42.879888 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 19:32:42.880193 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 19:32:42.881335 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.864283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.29.233.188:40533)
0810 19:32:42.864453 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.864459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.864478 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:32:42.864570 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.864575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.864681 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.865751 (+ 1070us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.865946 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.867126 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.868859 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.869348 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.869488 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.873061 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.873068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.873073 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.873118 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.876995 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.877901 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.877907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.877911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.878055 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.878565 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.878572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.878578 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.878951 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.879110 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.880886 (+ 1776us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.881067 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:32:42.883127 5023 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.233.188:40533: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "519617e664494d928d11e37fb078ba8a"
member_type: NON_VOTER
last_known_addr {
host: "127.29.233.188"
port: 40533
}
attrs {
promote: true
}
}
dest_uuid: "cfd2beedb5444b96b423800774185423"
cas_config_opid_index: 5
I20260810 19:32:42.884476 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.864335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.29.233.129:39067)
0810 19:32:42.865060 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.865067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.865091 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:32:42.865151 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.865157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.865375 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.865577 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.866946 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.868542 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.870474 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.871386 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.871542 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.876519 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.876531 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.876546 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.876584 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:42.880209 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.881408 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.881419 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.881421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.881592 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:42.882055 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:42.882059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:42.882062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:42.882712 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:42.883027 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:42.883710 (+ 683us) server_negotiation.cc:299] Negotiation successful
0810 19:32:42.884048 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":382,"threads_started":1}
I20260810 19:32:42.884404 5023 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } }
I20260810 19:32:42.885960 5223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:42.886556 5804 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 519617e664494d928d11e37fb078ba8a (127.29.233.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.891472 5679 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 LEARNER]: Refusing update from remote peer cfd2beedb5444b96b423800774185423: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 19:32:42.893537 5767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:42.894837 5828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 19:32:42.897434 5767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Refusing update from remote peer cfd2beedb5444b96b423800774185423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 19:32:42.903685 5679 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 519617e664494d928d11e37fb078ba8a (127.29.233.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.905440 5832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:42.905869 5804 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.233.188:40533
I20260810 19:32:42.906613 5629 master.cc:537] Master@127.29.233.188:40533 shutting down...
I20260810 19:32:42.907197 5804 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.907627 5804 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:42.916373 5707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: NON_VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: true } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.916703 5707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: LEARNER
I20260810 19:32:42.917395 5804 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: attempting to promote NON_VOTER 7e35c1c720704f8e9fd1433d2bfc6fb1 to VOTER
I20260810 19:32:42.919771 5804 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } }
I20260810 19:32:42.923487 5678 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 LEARNER]: Refusing update from remote peer cfd2beedb5444b96b423800774185423: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 19:32:42.924139 5828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260810 19:32:42.924994 5007 consensus_peers.cc:487] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 -> Peer 519617e664494d928d11e37fb078ba8a (127.29.233.188:40533): Couldn't send request to peer 519617e664494d928d11e37fb078ba8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 19:32:42.926990 5828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7e35c1c720704f8e9fd1433d2bfc6fb1 (127.29.233.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.931506 5832 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.931840 5832 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:42.932209 5832 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.932454 5832 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:42.931892 5679 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7e35c1c720704f8e9fd1433d2bfc6fb1 (127.29.233.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.937129 5707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:42.937390 5707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:32:42.944994 5629 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:42.945487 5629 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:42.945724 5629 tablet_replica.cc:330] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: stopping tablet replica
I20260810 19:32:42.963853 5629 master.cc:552] Master@127.29.233.188:40533 shutdown complete.
I20260810 19:32:42.964087 5629 master_runner.cc:305] Clearing existing system tablet
I20260810 19:32:42.964509 5629 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:42.964644 5629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:42.968647 5629 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260810 19:32:42.970372 5836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:42.970754 5629 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:42.971033 5629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 19:32:42.971133 5629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
uuid: "519617e664494d928d11e37fb078ba8a"
format_stamp: "Formatted at 2026-08-10 19:32:42 on dist-test-slave-tm1g"
server_key: "235dfac3907c4df337e94a3952e494cf"
server_key_iv: "ebe8766982cfa1a5b5ebc05e75fabddd"
server_key_version: "encryptionkey@0"
I20260810 19:32:42.976253 5629 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:42.983381 5629 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 19:32:42.983538 5629 log.cc:1196] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 19:32:42.984025 5629 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Deleting consensus metadata
I20260810 19:32:42.984268 5629 master_runner.cc:315] Copying system tablet from 127.29.233.190:46487
I20260810 19:32:42.986999 5629 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: tablet copy: Beginning tablet copy session from remote peer at address 127.29.233.190:46487
I20260810 19:32:43.005667 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.987860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.29.233.190:46487)
0810 19:32:42.988101 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:42.988107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:42.991165 (+ 3058us) server_negotiation.cc:407] Connection header received
0810 19:32:42.991342 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:42.991348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:42.991439 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:42.991553 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:42.992642 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.993797 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.995717 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.996399 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.996527 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.998971 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:42.998977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:42.998982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:42.999034 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:43.002657 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.003427 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.003433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.003437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.003610 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.004054 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.004059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.004063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.004421 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:43.004571 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:43.005062 (+ 491us) server_negotiation.cc:299] Negotiation successful
0810 19:32:43.005316 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260810 19:32:43.005700 5842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:42.987803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:59688)
0810 19:32:42.991019 (+ 3216us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:42.991040 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:42.991202 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:42.991594 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:42.991600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:42.992181 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:42.992470 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.992484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.993973 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.993979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.995549 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:42.995564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:42.996561 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:42.996567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:42.996764 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:42.997557 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:42.997579 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:42.998743 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:43.002805 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.002810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.002815 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.003290 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.003730 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.003735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.003739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.003911 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.004548 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:43.004553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:43.004738 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 19:32:43.004897 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 19:32:43.005231 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3046,"thread_start_us":2854,"threads_started":1}
I20260810 19:32:43.006721 5044 tablet_copy_service.cc:140] P cfd2beedb5444b96b423800774185423: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 519617e664494d928d11e37fb078ba8a ({username='oryx', principal='oryx/127.29.233.188@KRBTEST.COM'} at 127.0.0.1:59688)
I20260810 19:32:43.006917 5044 tablet_copy_service.cc:161] P cfd2beedb5444b96b423800774185423: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 519617e664494d928d11e37fb078ba8a at {username='oryx', principal='oryx/127.29.233.188@KRBTEST.COM'} at 127.0.0.1:59688: session id = 519617e664494d928d11e37fb078ba8a-00000000000000000000000000000000
I20260810 19:32:43.010777 5044 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 19:32:43.012704 5629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 19:32:43.019587 5629 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: tablet copy: Starting download of 0 data blocks...
I20260810 19:32:43.019944 5629 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: tablet copy: Starting download of 1 WAL segments...
I20260810 19:32:43.022641 5629 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 19:32:43.026944 5044 tablet_copy_service.cc:342] P cfd2beedb5444b96b423800774185423: Request end of tablet copy session 519617e664494d928d11e37fb078ba8a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.233.188@KRBTEST.COM'} at 127.0.0.1:59688
I20260810 19:32:43.027118 5044 tablet_copy_service.cc:433] P cfd2beedb5444b96b423800774185423: ending tablet copy session 519617e664494d928d11e37fb078ba8a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 519617e664494d928d11e37fb078ba8a
I20260810 19:32:43.039994 5629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:43.042851 5843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:43.044070 5844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:43.050920 5629 server_base.cc:1034] running on GCE node
W20260810 19:32:43.051760 5846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:43.052040 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 5089
I20260810 19:32:43.052371 5629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:43.058552 5629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:43.059850 5629 hybrid_clock.cc:648] HybridClock initialized: now 1786390363059795 us; error 47 us; skew 500 ppm
I20260810 19:32:43.060166 5629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:43 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390363, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:43.062579 5629 init.cc:377] Logged in from keytab as oryx/127.29.233.188@KRBTEST.COM (short username oryx)
I20260810 19:32:43.070670 5629 webserver.cc:460] Webserver started at http://127.29.233.188:45885/ using document root <none> and password file <none>
I20260810 19:32:43.071092 5629 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:43.071208 5629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:43.075003 5629 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 19:32:43.077329 5853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:43.077826 5629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 19:32:43.078025 5629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
uuid: "519617e664494d928d11e37fb078ba8a"
format_stamp: "Formatted at 2026-08-10 19:32:42 on dist-test-slave-tm1g"
server_key: "235dfac3907c4df337e94a3952e494cf"
server_key_iv: "ebe8766982cfa1a5b5ebc05e75fabddd"
server_key_version: "encryptionkey@0"
I20260810 19:32:43.078352 5629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:43.083312 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:39067
--local_ip_for_outbound_sockets=127.29.233.129
--tserver_master_addrs=127.29.233.190:46487
--webserver_port=33359
--webserver_interface=127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab
--principal=oryx/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:32:43.115314 5629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:43.119796 5629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:43.155303 5629 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.188:40533
I20260810 19:32:43.155388 5909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.188:40533 every 8 connection(s)
I20260810 19:32:43.156441 5629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
I20260810 19:32:43.162252 5910 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:32:43.166357 5910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Bootstrap starting.
I20260810 19:32:43.186986 5910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:32:43.187593 5910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: Bootstrap complete.
I20260810 19:32:43.188867 5910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } }
I20260810 19:32:43.189196 5910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 519617e664494d928d11e37fb078ba8a, State: Initialized, Role: LEARNER
I20260810 19:32:43.189474 5910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } }
I20260810 19:32:43.190517 5913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:43.191023 5913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: This master's current role is: LEARNER
I20260810 19:32:43.191329 5910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:43.206382 5927 catalog_manager.cc:1259] Loaded cluster ID: a9a7add4949e428395feb7252859a6ac
I20260810 19:32:43.206499 5927 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: loading cluster ID for follower catalog manager: success
I20260810 19:32:43.210479 5927 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: acquiring CA information for follower catalog manager: success
I20260810 19:32:43.212950 5927 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 19:32:43.473538 5858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:43.473860 5858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:43.474042 5858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:43.481792 5858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:43.481945 5858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:43.482121 5858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:43.493419 5858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab
--principal=oryx/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:39067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=33359
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:43.494848 5858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:43.496183 5858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:43.510015 5933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:43.510139 5932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:43.510094 5935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:43.515398 5858 server_base.cc:1034] running on GCE node
I20260810 19:32:43.516085 5858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:43.517145 5858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:43.518443 5858 hybrid_clock.cc:648] HybridClock initialized: now 1786390363518395 us; error 41 us; skew 500 ppm
I20260810 19:32:43.519627 5858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:43 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390363, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:43.523540 5858 init.cc:377] Logged in from keytab as oryx/127.29.233.129@KRBTEST.COM (short username oryx)
I20260810 19:32:43.526657 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.518885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.0.0.1:34704)
0810 19:32:43.519724 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:43.519746 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:43.519926 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:43.520427 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:43.520435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:43.520471 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:43.520721 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.520734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.522343 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.522350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.523725 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.523739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.525940 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.525944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.526078 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.526103 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 19:32:43.526169 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 19:32:43.526276 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":596,"threads_started":1}
I20260810 19:32:43.526958 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.518989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.29.233.188:40533)
0810 19:32:43.519545 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:43.519552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:43.519838 (+ 286us) server_negotiation.cc:407] Connection header received
0810 19:32:43.520117 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:43.520124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:43.520232 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:43.520378 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:43.521187 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.522152 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.523930 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.525749 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.525984 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.526241 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:43.526346 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 19:32:43.526513 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":280,"threads_started":1}
I20260810 19:32:43.527868 5883 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 19:32:43.528385 5858 webserver.cc:460] Webserver started at http://127.29.233.129:33359/ using document root <none> and password file <none>
I20260810 19:32:43.528517 5913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd2beedb5444b96b423800774185423. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:43.528833 5913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: This master's current role is: LEARNER
I20260810 19:32:43.529354 5858 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:43.529546 5858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:43.535264 5858 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 19:32:43.537942 5946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:43.539538 5858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:32:43.539733 5858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "b03dfb8ab4034d3483dd6db7b65bffe6"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "35ff038f609246c1b1d478c31a8b5301"
server_key_iv: "2cd657778ecd3626366e0678d2b83a25"
server_key_version: "encryptionkey@0"
I20260810 19:32:43.540800 5858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:43.560261 5858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:43.564846 5858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:43.565153 5858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:43.566222 5858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:43.567715 5858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:43.567839 5858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:43.567981 5858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:43.568074 5858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:43.603106 5858 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39067
I20260810 19:32:43.603191 6064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39067 every 8 connection(s)
I20260810 19:32:43.604959 5858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:43.605661 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 5858
I20260810 19:32:43.606055 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 5234
Aug 10 19:32:43 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390363, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.188@KRBTEST.COM
Aug 10 19:32:43 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390363, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
Aug 10 19:32:43 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.130@KRBTEST.COM
I20260810 19:32:43.640688 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.610577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:51247 (local address 127.29.233.188:40533)
0810 19:32:43.610764 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:43.610770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:43.620277 (+ 9507us) server_negotiation.cc:407] Connection header received
0810 19:32:43.620456 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:43.620462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:43.620550 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:43.620656 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:43.622971 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.623706 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.626132 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.627118 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.627227 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.632445 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:43.632455 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:43.632464 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:43.632514 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:43.636100 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.637326 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.637336 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.637342 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.637547 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.638159 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.638165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.638170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.638628 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:43.638787 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:43.639877 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0810 19:32:43.640140 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 19:32:43.641820 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.609858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.29.233.129:51247)
0810 19:32:43.620127 (+ 10269us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:43.620152 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:43.620318 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:43.621189 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:43.621199 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:43.621924 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:43.622764 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.622795 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.623889 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.623895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.625952 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.625966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.627621 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.627629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.628025 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.628897 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:43.628942 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:43.632137 (+ 3195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:43.636234 (+ 4097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.636247 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.636274 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.637126 (+ 852us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.637708 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.637715 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.637722 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.638025 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.638784 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:43.638800 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:43.639335 (+ 535us) client_negotiation.cc:769] Sending connection context
0810 19:32:43.639678 (+ 343us) client_negotiation.cc:240] Negotiation successful
0810 19:32:43.640244 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10094,"thread_start_us":10557,"threads_started":1}
I20260810 19:32:43.643694 6065 heartbeater.cc:347] Connected to a master server at 127.29.233.188:40533
I20260810 19:32:43.644083 6065 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:43.645045 6065 heartbeater.cc:511] Master 127.29.233.188:40533 requested a full tablet report, sending...
I20260810 19:32:43.646103 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.607593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.29.233.130:42911)
0810 19:32:43.608351 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:43.608363 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:43.615004 (+ 6641us) server_negotiation.cc:407] Connection header received
0810 19:32:43.615185 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:43.615193 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:43.615410 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:43.615639 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:43.617246 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.618297 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.619747 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.620683 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.620811 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.635899 (+ 15088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:43.635922 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:43.635942 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:43.636006 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:43.639919 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.643379 (+ 3460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.643388 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.643391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.643580 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.644092 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.644097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.644101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.644536 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:43.644849 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:43.645396 (+ 547us) server_negotiation.cc:299] Negotiation successful
0810 19:32:43.645682 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":391,"threads_started":1}
I20260810 19:32:43.647845 5364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:43.647790 5872 ts_manager.cc:194] Registered new tserver with Master: b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129:39067)
Aug 10 19:32:43 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390363, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.189@KRBTEST.COM
I20260810 19:32:43.654670 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.608118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53369 (local address 127.29.233.190:46487)
0810 19:32:43.624070 (+ 15952us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:43.624079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:43.624101 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:32:43.624171 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:43.624177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:43.624264 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:43.624398 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:43.625455 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.627026 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.628975 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.629861 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.630029 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.636367 (+ 6338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:43.636374 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:43.636379 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:43.636421 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:43.642343 (+ 5922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.646499 (+ 4156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.646505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.646508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.646684 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.652902 (+ 6218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.652919 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.652925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.653356 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:43.653512 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:43.653929 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 19:32:43.654188 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15781,"thread_start_us":16476,"threads_started":1}
I20260810 19:32:43.654680 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.608978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:53369)
0810 19:32:43.610231 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:43.610329 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:43.613753 (+ 3424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:43.624452 (+ 10699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:43.624457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:43.624924 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:43.625236 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.625248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.627170 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.627179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.628822 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.628836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.630107 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.630115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.630373 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.631084 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:43.631109 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:43.634423 (+ 3314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:43.642747 (+ 8324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.642754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.642759 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.644022 (+ 1263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.647334 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.647341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.647347 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.647601 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.653483 (+ 5882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:43.653488 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:43.653648 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 19:32:43.653809 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 19:32:43.654126 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"spinlock_wait_cycles":2432,"thread_start_us":1233,"threads_started":1}
I20260810 19:32:43.655953 6066 heartbeater.cc:347] Connected to a master server at 127.29.233.190:46487
I20260810 19:32:43.656123 6066 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:43.656612 6066 heartbeater.cc:511] Master 127.29.233.190:46487 requested a full tablet report, sending...
I20260810 19:32:43.657166 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.625731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.29.233.129:48153)
0810 19:32:43.639446 (+ 13715us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:43.639468 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:43.639861 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:43.640737 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:43.640742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:43.641340 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:43.641674 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.641685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.643595 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.643599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.645279 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:43.645314 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.646743 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.646749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:43.646924 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.647533 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:43.647556 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:43.650359 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:43.654079 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.654086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.654090 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.654669 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.655183 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:43.655188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:43.655192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:43.655355 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:43.656083 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:43.656088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:43.656240 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 19:32:43.656375 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 19:32:43.656681 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13565,"thread_start_us":13841,"threads_started":1}
I20260810 19:32:43.657979 5023 ts_manager.cc:194] Re-registered known tserver with Master: b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129:39067)
I20260810 19:32:43.658133 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:43.625951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:48153 (local address 127.29.233.189:34267)
0810 19:32:43.633346 (+ 7395us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:43.633353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:43.639557 (+ 6204us) server_negotiation.cc:407] Connection header received
0810 19:32:43.640010 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:43.640017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:43.640199 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:43.640347 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:43.641864 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.643379 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.645489 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:43.646217 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:43.646359 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:43.650628 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:43.650634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:43.650639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:43.650679 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:43.653946 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.654810 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.654816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.654819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.655029 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:43.655478 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:43.655484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:43.655487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:43.655850 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:43.655997 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:43.657465 (+ 1468us) server_negotiation.cc:299] Negotiation successful
0810 19:32:43.657712 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2702,"thread_start_us":2977,"threads_started":1}
I20260810 19:32:43.658937 5023 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:53369
I20260810 19:32:43.660151 6067 heartbeater.cc:347] Connected to a master server at 127.29.233.189:34267
I20260810 19:32:43.660360 6067 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:43.660851 6067 heartbeater.cc:511] Master 127.29.233.189:34267 requested a full tablet report, sending...
I20260810 19:32:43.662611 5669 ts_manager.cc:194] Registered new tserver with Master: b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129:39067)
I20260810 19:32:43.736896 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 5234
I20260810 19:32:43.759442 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:42911
--local_ip_for_outbound_sockets=127.29.233.130
--tserver_master_addrs=127.29.233.190:46487
--webserver_port=36525
--webserver_interface=127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab
--principal=oryx/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:32:44.025341 5939 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: NON_VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: true } } }
I20260810 19:32:44.025852 5939 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:44.036412 6079 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: attempting to promote NON_VOTER 519617e664494d928d11e37fb078ba8a to VOTER
I20260810 19:32:44.037115 6079 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } }
I20260810 19:32:44.039507 5883 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 LEARNER]: Refusing update from remote peer cfd2beedb5444b96b423800774185423: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 19:32:44.039592 5679 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Refusing update from remote peer cfd2beedb5444b96b423800774185423: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 19:32:44.040234 6080 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 19:32:44.040668 6081 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 19:32:44.043684 6080 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 519617e664494d928d11e37fb078ba8a (127.29.233.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.046859 6080 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.047104 6080 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:44.047367 6081 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.047679 6081 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:44.048134 5679 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 519617e664494d928d11e37fb078ba8a (127.29.233.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.054350 6087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.054805 6087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:32:44.058429 6080 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.058725 6080 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:44.059183 5883 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 519617e664494d928d11e37fb078ba8a (127.29.233.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.062569 6088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cfd2beedb5444b96b423800774185423" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:44.062956 6088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: This master's current role is: FOLLOWER
W20260810 19:32:44.126120 6078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:44.126495 6078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:44.126603 6078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:44.134101 6078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:44.134225 6078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:44.134430 6078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:44.144801 6078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab
--principal=oryx/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:42911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=36525
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:44.146273 6078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:44.147724 6078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:44.162670 6094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:44.162649 6095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:44.164171 6078 server_base.cc:1034] running on GCE node
W20260810 19:32:44.166581 6097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:44.167285 6078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:44.168335 6078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:44.169612 6078 hybrid_clock.cc:648] HybridClock initialized: now 1786390364169544 us; error 63 us; skew 500 ppm
I20260810 19:32:44.169978 6078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:44.173244 6078 init.cc:377] Logged in from keytab as oryx/127.29.233.130@KRBTEST.COM (short username oryx)
I20260810 19:32:44.175098 6078 webserver.cc:460] Webserver started at http://127.29.233.130:36525/ using document root <none> and password file <none>
I20260810 19:32:44.175836 6078 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:44.176005 6078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:44.181620 6078 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 19:32:44.184454 6106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:44.185807 6078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:44.185992 6078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "7006b47b0979434d9683a6f340627ee4"
format_stamp: "Formatted at 2026-08-10 19:32:40 on dist-test-slave-tm1g"
server_key: "b1b7bde15bf6e00bf0c9a19332fb0292"
server_key_iv: "d3277acf20ed50cf8bd59c5d1d74e170"
server_key_version: "encryptionkey@0"
I20260810 19:32:44.187062 6078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:44.228735 6078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:44.233000 6078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:44.233330 6078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:44.234510 6078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:44.235931 6078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:44.236060 6078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:44.236197 6078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:44.236289 6078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:44.271049 6078 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:42911
I20260810 19:32:44.271139 6227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:42911 every 8 connection(s)
I20260810 19:32:44.272653 6078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:44.275209 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 6078
I20260810 19:32:44.275532 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 5376
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.131@KRBTEST.COM
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.189@KRBTEST.COM
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.188@KRBTEST.COM
I20260810 19:32:44.313580 6235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.277162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:50557 (local address 127.29.233.188:40533)
0810 19:32:44.288893 (+ 11731us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.288899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.288920 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:32:44.291033 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.291043 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.291125 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.291295 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.295122 (+ 3827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.296340 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.299751 (+ 3411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.300556 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.300693 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.306580 (+ 5887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.306588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.306594 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.306643 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.310221 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.311136 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.311141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.311144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.311290 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.311743 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.311749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.311751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.312097 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.312247 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.312949 (+ 702us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.313165 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11577,"thread_start_us":11348,"threads_started":1}
I20260810 19:32:44.314411 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.275886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:32837 (local address 127.29.233.190:46487)
0810 19:32:44.282768 (+ 6882us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.282773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.288346 (+ 5573us) server_negotiation.cc:407] Connection header received
0810 19:32:44.290458 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.290466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.290580 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.290781 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.293742 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.294914 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.297768 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.298551 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.298693 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.305489 (+ 6796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.305497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.305502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.305562 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.309734 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.310753 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.310759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.310763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.311996 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.312566 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.312572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.312575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.312966 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.313110 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.313748 (+ 638us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.313991 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6758,"thread_start_us":6604,"threads_started":1}
I20260810 19:32:44.314527 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.276363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.29.233.130:50557)
0810 19:32:44.287060 (+ 10697us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:44.287116 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:44.290092 (+ 2976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:44.291338 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:44.291344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:44.292412 (+ 1068us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:44.293527 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.293557 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.296551 (+ 2994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.296556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.299550 (+ 2994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.299567 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.301001 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.301009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.301211 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.301890 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:44.301924 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:44.305318 (+ 3394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:44.310384 (+ 5066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.310392 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.310397 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.310997 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.311429 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.311434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.311437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.311617 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.312312 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:44.312320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:44.312543 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 19:32:44.312670 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 19:32:44.313929 (+ 1259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10335,"mutex_wait_us":1082,"spinlock_wait_cycles":32512,"thread_start_us":10080,"threads_started":1}
I20260810 19:32:44.315421 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.288047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:42317 (local address 127.29.233.189:34267)
0810 19:32:44.289848 (+ 1801us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.289853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.289870 (+ 17us) server_negotiation.cc:407] Connection header received
0810 19:32:44.290614 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.290621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.290709 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.290835 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.293782 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.294690 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.297806 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.298223 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.298410 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.304758 (+ 6348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.304768 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.304774 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.304818 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.308737 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.309971 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.309978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.309985 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.310144 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.310882 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.310888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.310892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.311300 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.311438 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.314684 (+ 3246us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.314971 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1625,"thread_start_us":1495,"threads_started":1}
I20260810 19:32:44.315472 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.278326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.130:32837)
0810 19:32:44.288232 (+ 9906us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:44.288257 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:44.290107 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:44.291071 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:44.291084 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:44.292418 (+ 1334us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:44.293599 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.293608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.295053 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.295058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.297645 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.297656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.298819 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.298827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.299219 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.300201 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:44.300235 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:44.305265 (+ 5030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:44.309900 (+ 4635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.309905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.309909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.310526 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.312210 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.312215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.312220 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.312399 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.313210 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:44.313217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:44.313392 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 19:32:44.314693 (+ 1301us) client_negotiation.cc:240] Negotiation successful
0810 19:32:44.315011 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9761,"mutex_wait_us":435,"spinlock_wait_cycles":61952,"thread_start_us":8234,"threads_started":1}
I20260810 19:32:44.314496 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.288437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.29.233.130:42317)
0810 19:32:44.289230 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:44.289257 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:44.290107 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:44.291071 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:44.291084 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:44.292439 (+ 1355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:44.293572 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.293592 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.294859 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.294865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.297601 (+ 2736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.297629 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.298831 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.298838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.299214 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.300191 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:44.300225 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:44.304416 (+ 4191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:44.308921 (+ 4505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.308932 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.308965 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.309793 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.310411 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.310415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.310418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.310737 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.311524 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:44.311532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:44.312010 (+ 478us) client_negotiation.cc:769] Sending connection context
0810 19:32:44.312369 (+ 359us) client_negotiation.cc:240] Negotiation successful
0810 19:32:44.312972 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":431,"spinlock_wait_cycles":2432,"thread_start_us":476,"threads_started":1}
I20260810 19:32:44.316593 6228 heartbeater.cc:347] Connected to a master server at 127.29.233.188:40533
I20260810 19:32:44.316825 6230 heartbeater.cc:347] Connected to a master server at 127.29.233.189:34267
I20260810 19:32:44.316985 6230 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:44.316989 6228 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:44.317844 6236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.277734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.29.233.131:45957)
0810 19:32:44.287180 (+ 9446us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.287192 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.287225 (+ 33us) server_negotiation.cc:407] Connection header received
0810 19:32:44.287296 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.287304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.287590 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.287758 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.291885 (+ 4127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.293586 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.295527 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.296372 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.296496 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.305908 (+ 9412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.305924 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.305938 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.305974 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.311589 (+ 5615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.312666 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.312675 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.312678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.312834 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.313316 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.313321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.313324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.314155 (+ 831us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.316875 (+ 2720us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.317121 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.317404 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9270,"thread_start_us":9079,"threads_started":1}
I20260810 19:32:44.317981 6229 heartbeater.cc:347] Connected to a master server at 127.29.233.190:46487
I20260810 19:32:44.318120 6229 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:44.318131 6228 heartbeater.cc:511] Master 127.29.233.188:40533 requested a full tablet report, sending...
I20260810 19:32:44.318599 6229 heartbeater.cc:511] Master 127.29.233.190:46487 requested a full tablet report, sending...
I20260810 19:32:44.318874 6230 heartbeater.cc:511] Master 127.29.233.189:34267 requested a full tablet report, sending...
I20260810 19:32:44.319294 5508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:44.320057 5023 ts_manager.cc:194] Re-registered known tserver with Master: 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911)
I20260810 19:32:44.320231 5669 ts_manager.cc:194] Registered new tserver with Master: 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911)
I20260810 19:32:44.321329 5023 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:32837
I20260810 19:32:44.323113 5872 ts_manager.cc:194] Registered new tserver with Master: 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911)
I20260810 19:32:44.407642 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 5376
I20260810 19:32:44.428544 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:45957
--local_ip_for_outbound_sockets=127.29.233.131
--tserver_master_addrs=127.29.233.190:46487
--webserver_port=33551
--webserver_interface=127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab
--principal=oryx/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:32827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:32:44.661550 6066 heartbeater.cc:503] Master 127.29.233.190:46487 was elected leader, sending a full tablet report...
W20260810 19:32:44.787981 6241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:44.788268 6241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:44.788403 6241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:44.796084 6241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:44.796197 6241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:44.796321 6241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:44.806578 6241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:32827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab
--principal=oryx/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:45957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=33551
--tserver_master_addrs=127.29.233.190:46487,127.29.233.189:34267,127.29.233.188:40533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:44.807943 6241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:44.809422 6241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:44.823828 6247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:44.823958 6248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:44.823967 6250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:44.824869 6241 server_base.cc:1034] running on GCE node
I20260810 19:32:44.825606 6241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:44.826761 6241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:44.828004 6241 hybrid_clock.cc:648] HybridClock initialized: now 1786390364827942 us; error 59 us; skew 500 ppm
I20260810 19:32:44.828413 6241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:44.831810 6241 init.cc:377] Logged in from keytab as oryx/127.29.233.131@KRBTEST.COM (short username oryx)
I20260810 19:32:44.833424 6241 webserver.cc:460] Webserver started at http://127.29.233.131:33551/ using document root <none> and password file <none>
I20260810 19:32:44.834208 6241 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:44.834409 6241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:44.840364 6241 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 19:32:44.843693 6257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:44.845361 6241 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 19:32:44.845556 6241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "f7f2920c516d4db2b3a85bda54bbbd8c"
format_stamp: "Formatted at 2026-08-10 19:32:41 on dist-test-slave-tm1g"
server_key: "31dca7a38fa9574a3b3cf9c7c7d9125a"
server_key_iv: "29fb7a3d981bbec3de32c72cd27367c7"
server_key_version: "encryptionkey@0"
I20260810 19:32:44.846735 6241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:44.869885 6241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:44.873952 6241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:44.874279 6241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:44.875428 6241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:44.876910 6241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:44.877040 6241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:44.877180 6241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:44.877287 6241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:44.913192 6241 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:45957
I20260810 19:32:44.913317 6376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:45957 every 8 connection(s)
I20260810 19:32:44.914801 6241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:44.917541 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 6241
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.188@KRBTEST.COM
I20260810 19:32:44.953742 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.919395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:37005 (local address 127.29.233.188:40533)
0810 19:32:44.923301 (+ 3906us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.923310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.928031 (+ 4721us) server_negotiation.cc:407] Connection header received
0810 19:32:44.928130 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.928139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.928287 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.928408 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.931846 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.933461 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.936881 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.937801 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.937925 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.944535 (+ 6610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.944543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.944548 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.944597 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.948961 (+ 4364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.950124 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.950133 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.950139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.950408 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.951039 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.951047 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.951053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.951595 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.951752 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.952918 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.953206 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3758,"thread_start_us":3641,"threads_started":1}
I20260810 19:32:44.956988 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.918709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.29.233.131:37005)
0810 19:32:44.923787 (+ 5078us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:44.923856 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:44.927512 (+ 3656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:44.929075 (+ 1563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:44.929093 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:44.930229 (+ 1136us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:44.931373 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.931405 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.933739 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.933749 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.936650 (+ 2901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.936672 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.938399 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.938409 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.938926 (+ 517us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.939997 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:44.940043 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:44.944198 (+ 4155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:44.949200 (+ 5002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.949214 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.949237 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.949963 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.950579 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.950586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.950592 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.950893 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.951863 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:44.951872 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:44.952415 (+ 543us) client_negotiation.cc:769] Sending connection context
0810 19:32:44.954521 (+ 2106us) client_negotiation.cc:240] Negotiation successful
0810 19:32:44.955216 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4635,"thread_start_us":4405,"threads_started":1}
I20260810 19:32:44.959059 6377 heartbeater.cc:347] Connected to a master server at 127.29.233.188:40533
I20260810 19:32:44.959462 6377 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:44.960471 6377 heartbeater.cc:511] Master 127.29.233.188:40533 requested a full tablet report, sending...
I20260810 19:32:44.962659 5872 ts_manager.cc:194] Registered new tserver with Master: f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957)
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
Aug 10 19:32:44 dist-test-slave-tm1g krb5kdc[4976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390364, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.189@KRBTEST.COM
I20260810 19:32:44.982700 6385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.920012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.131:35791)
0810 19:32:44.951848 (+ 31836us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:44.951870 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:44.952002 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:44.954050 (+ 2048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:44.954058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:44.954813 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:44.955141 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.955158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.962071 (+ 6913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.962080 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.964658 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.964677 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.966369 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.966378 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.966648 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.967750 (+ 1102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:44.967775 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:44.974282 (+ 6507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:44.979038 (+ 4756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.979045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.979049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.979622 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.980456 (+ 834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.980459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.980462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.980579 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.981597 (+ 1018us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:44.981602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:44.981772 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 19:32:44.981926 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 19:32:44.982164 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31699,"thread_start_us":32057,"threads_started":1}
I20260810 19:32:44.982746 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.919540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:35791 (local address 127.29.233.190:46487)
0810 19:32:44.953147 (+ 33607us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.953164 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.953193 (+ 29us) server_negotiation.cc:407] Connection header received
0810 19:32:44.953274 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.953283 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.953391 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.953529 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.955919 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.957368 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.964853 (+ 7485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.965737 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.965893 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.974518 (+ 8625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.974527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.974532 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.974575 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.978862 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.979758 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.979766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.979773 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.979941 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.980709 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.980717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.980723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.981113 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.981258 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.982045 (+ 787us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.982348 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24719,"thread_start_us":31212,"threads_started":1}
I20260810 19:32:44.983644 6378 heartbeater.cc:347] Connected to a master server at 127.29.233.190:46487
I20260810 19:32:44.983803 6378 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:44.984300 6378 heartbeater.cc:511] Master 127.29.233.190:46487 requested a full tablet report, sending...
I20260810 19:32:44.985415 5023 ts_manager.cc:194] Re-registered known tserver with Master: f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957)
I20260810 19:32:44.986631 5023 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:35791
I20260810 19:32:44.992743 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.945196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.29.233.188:40533)
0810 19:32:44.954433 (+ 9237us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.954441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.968473 (+ 14032us) server_negotiation.cc:407] Connection header received
0810 19:32:44.968882 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.968889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.968982 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.969055 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.971443 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.972552 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.980779 (+ 8227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.981415 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.981521 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.985109 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.985115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.985119 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.985155 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.988901 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.989731 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.989737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.989741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.989920 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.990407 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.990414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.990417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.990774 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.991027 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.991953 (+ 926us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.992355 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9080,"thread_start_us":20980,"threads_started":1}
I20260810 19:32:44.992861 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.952249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.29.233.189:34267)
0810 19:32:44.967904 (+ 15655us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.967912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.969533 (+ 1621us) server_negotiation.cc:407] Connection header received
0810 19:32:44.970542 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.970550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.970642 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.970723 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.973292 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.974629 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.976767 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.977456 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.977560 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.981452 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.981462 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.981469 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.981521 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.987295 (+ 5774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.989273 (+ 1978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.989279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.989284 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.989458 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.989928 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.989934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.989937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.990293 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.990601 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.991244 (+ 643us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.992500 (+ 1256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15517,"thread_start_us":20334,"threads_started":1}
I20260810 19:32:44.993995 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.944673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.29.233.190:46487)
0810 19:32:44.961878 (+ 17205us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.961887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.967013 (+ 5126us) server_negotiation.cc:407] Connection header received
0810 19:32:44.967209 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.967217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.967316 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.967444 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.970386 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.971638 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.977587 (+ 5949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.978268 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.978412 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.986650 (+ 8238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.986656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.986661 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.986710 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.990478 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.991392 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.991398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.991402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.991600 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.992084 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.992090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.992094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.992462 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.992748 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.993343 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.993594 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17028,"thread_start_us":17228,"threads_started":1}
I20260810 19:32:44.995507 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.952623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.29.233.131:58433)
0810 19:32:44.958618 (+ 5995us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:44.958636 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:44.958727 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:44.972126 (+ 13399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:44.972131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:44.972676 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:44.972917 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.972929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.974121 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.974129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.975637 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:44.975652 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.976695 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.976703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:44.976917 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.977446 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:44.977471 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:44.981406 (+ 3935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:44.992147 (+ 10741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.992153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.992157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.992678 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.993525 (+ 847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:44.993530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:44.993534 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:44.993693 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:44.994617 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:44.994622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:44.994777 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 19:32:44.994913 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 19:32:44.995236 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5844,"thread_start_us":19133,"threads_started":1}
I20260810 19:32:44.996425 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:44.952664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:58433 (local address 127.29.233.189:34267)
0810 19:32:44.971797 (+ 19133us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:44.971804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:44.971821 (+ 17us) server_negotiation.cc:407] Connection header received
0810 19:32:44.971870 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:44.971876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:44.971951 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:44.973069 (+ 1118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:44.973256 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.973985 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.975815 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:44.976511 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:44.977218 (+ 707us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:44.981658 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:44.981668 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:44.981676 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:44.981716 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:44.992005 (+ 10289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.993241 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.993247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.993249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.993388 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:44.993838 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:44.993843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:44.993846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:44.994468 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:44.995779 (+ 1311us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:44.995937 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 19:32:44.996167 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18997,"thread_start_us":15016,"threads_started":1}
I20260810 19:32:44.997186 6379 heartbeater.cc:347] Connected to a master server at 127.29.233.189:34267
I20260810 19:32:44.997342 6379 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:44.997833 6379 heartbeater.cc:511] Master 127.29.233.189:34267 requested a full tablet report, sending...
I20260810 19:32:44.999033 5669 ts_manager.cc:194] Registered new tserver with Master: f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957)
I20260810 19:32:44.999902 5023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:45.007135 5023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:45.040149 6409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.029774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33148 (local address 127.29.233.130:42911)
0810 19:32:45.030391 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.030402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.031950 (+ 1548us) server_negotiation.cc:407] Connection header received
0810 19:32:45.032214 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.032225 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.032509 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.032704 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:45.034360 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.035635 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.037453 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.038882 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.039037 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.039234 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.039684 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.039861 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":455,"threads_started":1}
I20260810 19:32:45.040124 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.029100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:42911 (local address 127.0.0.1:33148)
0810 19:32:45.031824 (+ 2724us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:45.031841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:45.032003 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:45.033312 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:45.033317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:45.033344 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:45.033992 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.034006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.035779 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.035783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.037284 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.037329 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.039256 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.039262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.039466 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.039516 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 19:32:45.039634 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 19:32:45.039765 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2621,"thread_start_us":2449,"threads_started":1}
I20260810 19:32:45.041749 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.029391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.29.233.129:39067)
0810 19:32:45.031269 (+ 1878us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.031281 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.032556 (+ 1275us) server_negotiation.cc:407] Connection header received
0810 19:32:45.032663 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.032668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.032828 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.033032 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:45.033996 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.037228 (+ 3232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.039555 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.040688 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.040860 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.041028 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.041267 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.041425 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1667,"thread_start_us":1320,"threads_started":1}
I20260810 19:32:45.041812 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.028723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39067 (local address 127.0.0.1:47834)
0810 19:32:45.032278 (+ 3555us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:45.032293 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:45.032430 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:45.033014 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:45.033019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:45.033049 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:45.033741 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.033759 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.037388 (+ 3629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.037397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.039414 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.039427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.040900 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.040908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.041081 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.041115 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 19:32:45.041225 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 19:32:45.041368 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2825,"spinlock_wait_cycles":4352,"thread_start_us":756,"threads_started":1}
I20260810 19:32:45.042836 6411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.030246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.29.233.131:45957)
0810 19:32:45.031016 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.031026 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.032410 (+ 1384us) server_negotiation.cc:407] Connection header received
0810 19:32:45.032569 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.032577 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.032789 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.032982 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:45.034044 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.035474 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.038094 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.041196 (+ 3102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.041403 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.041571 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.042339 (+ 768us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.042498 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":616,"threads_started":1}
I20260810 19:32:45.043462 6412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.029648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45957 (local address 127.0.0.1:33098)
0810 19:32:45.032270 (+ 2622us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:45.032290 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:45.032437 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:45.032996 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:45.033001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:45.033028 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:45.033741 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.033759 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.035618 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.035623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.037919 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.037934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.041718 (+ 3784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.041729 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.041945 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.041981 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 19:32:45.042070 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 19:32:45.043025 (+ 955us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2487,"mutex_wait_us":1921,"spinlock_wait_cycles":768,"thread_start_us":394,"threads_started":1}
I20260810 19:32:45.044787 6156 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba760b8f4b74757ab2db6093ab02831 (DEFAULT_TABLE table=test-table [id=a677c521dcfd47baa76a8e7a205ffd12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:45.045447 5996 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba760b8f4b74757ab2db6093ab02831 (DEFAULT_TABLE table=test-table [id=a677c521dcfd47baa76a8e7a205ffd12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:45.046396 6156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba760b8f4b74757ab2db6093ab02831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:45.046640 5996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba760b8f4b74757ab2db6093ab02831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:45.047032 6309 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba760b8f4b74757ab2db6093ab02831 (DEFAULT_TABLE table=test-table [id=a677c521dcfd47baa76a8e7a205ffd12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:45.048372 6309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba760b8f4b74757ab2db6093ab02831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:45.058799 6413 tablet_bootstrap.cc:492] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Bootstrap starting.
I20260810 19:32:45.061517 6414 tablet_bootstrap.cc:492] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Bootstrap starting.
I20260810 19:32:45.062371 6413 tablet_bootstrap.cc:654] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:45.063725 6413 log.cc:824] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:45.064644 6415 tablet_bootstrap.cc:492] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Bootstrap starting.
I20260810 19:32:45.065418 6414 tablet_bootstrap.cc:654] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:45.066913 6414 log.cc:824] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:45.068156 6415 tablet_bootstrap.cc:654] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:45.069087 6415 log.cc:824] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:45.069120 6413 tablet_bootstrap.cc:492] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: No bootstrap required, opened a new log
I20260810 19:32:45.069447 6413 ts_tablet_manager.cc:1397] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 19:32:45.069943 6414 tablet_bootstrap.cc:492] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: No bootstrap required, opened a new log
I20260810 19:32:45.070200 6414 ts_tablet_manager.cc:1397] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 19:32:45.071960 6415 tablet_bootstrap.cc:492] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: No bootstrap required, opened a new log
I20260810 19:32:45.072590 6415 ts_tablet_manager.cc:1397] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 19:32:45.079778 6415 raft_consensus.cc:348] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.080211 6415 raft_consensus.cc:374] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:45.080317 6415 raft_consensus.cc:729] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f2920c516d4db2b3a85bda54bbbd8c, State: Initialized, Role: FOLLOWER
I20260810 19:32:45.080371 6413 raft_consensus.cc:348] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.080801 6413 raft_consensus.cc:374] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:45.080927 6413 raft_consensus.cc:729] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03dfb8ab4034d3483dd6db7b65bffe6, State: Initialized, Role: FOLLOWER
I20260810 19:32:45.080813 6414 raft_consensus.cc:348] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.081019 6415 consensus_queue.cc:260] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.081322 6414 raft_consensus.cc:374] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:45.081432 6414 raft_consensus.cc:729] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7006b47b0979434d9683a6f340627ee4, State: Initialized, Role: FOLLOWER
I20260810 19:32:45.081518 6413 consensus_queue.cc:260] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.082021 6414 consensus_queue.cc:260] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.088389 6229 heartbeater.cc:503] Master 127.29.233.190:46487 was elected leader, sending a full tablet report...
I20260810 19:32:45.089394 6414 ts_tablet_manager.cc:1428] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260810 19:32:45.091044 6378 heartbeater.cc:503] Master 127.29.233.190:46487 was elected leader, sending a full tablet report...
I20260810 19:32:45.091925 6415 ts_tablet_manager.cc:1428] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Time spent starting tablet: real 0.019s user 0.007s sys 0.010s
I20260810 19:32:45.094380 6413 ts_tablet_manager.cc:1428] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Time spent starting tablet: real 0.025s user 0.018s sys 0.000s
W20260810 19:32:45.136972 6076 tablet.cc:2369] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:32:45.216631 6380 tablet.cc:2369] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:32:45.281656 6231 tablet.cc:2369] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:32:45.281554 6420 raft_consensus.cc:482] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:45.281937 6420 raft_consensus.cc:504] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.283702 6420 leader_election.cc:288] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911), f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957)
I20260810 19:32:45.291178 6409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.284409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53373 (local address 127.29.233.130:42911)
0810 19:32:45.284637 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.284646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.285351 (+ 705us) server_negotiation.cc:407] Connection header received
0810 19:32:45.285658 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.285664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.285741 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.285858 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:45.286569 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.287294 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.289191 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.289994 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.290179 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.290266 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.290682 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.290849 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:32:45.291190 6411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.284190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:36357 (local address 127.29.233.131:45957)
0810 19:32:45.284455 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.284463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.285270 (+ 807us) server_negotiation.cc:407] Connection header received
0810 19:32:45.285585 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.285593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.285672 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.285780 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:45.286415 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.287178 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.289199 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.290071 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.290266 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.290384 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.290769 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.290914 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:32:45.291290 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.284237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:42911 (local address 127.29.233.129:53373)
0810 19:32:45.285210 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:45.285224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:45.285413 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:45.285873 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:45.285877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:45.285912 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:45.286374 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.286390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.287451 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.287456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.289022 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.289035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.290322 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.290330 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.290486 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.290510 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:32:45.290638 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 19:32:45.290791 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":74,"thread_start_us":590,"threads_started":1}
I20260810 19:32:45.291370 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.284228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45957 (local address 127.29.233.129:36357)
0810 19:32:45.285153 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:45.285170 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:45.285361 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:45.285806 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:45.285811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:45.285837 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:45.286227 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.286241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.287325 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.287331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.289023 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:45.289043 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.290333 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.290339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:45.290569 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.290625 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 19:32:45.290725 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 19:32:45.290852 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":33,"thread_start_us":525,"threads_started":1}
I20260810 19:32:45.292498 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba760b8f4b74757ab2db6093ab02831" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" is_pre_election: true
I20260810 19:32:45.292728 6177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba760b8f4b74757ab2db6093ab02831" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7006b47b0979434d9683a6f340627ee4" is_pre_election: true
I20260810 19:32:45.293172 6331 raft_consensus.cc:2454] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 0.
I20260810 19:32:45.293195 6177 raft_consensus.cc:2454] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 0.
I20260810 19:32:45.293919 5948 leader_election.cc:302] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7006b47b0979434d9683a6f340627ee4, b03dfb8ab4034d3483dd6db7b65bffe6, f7f2920c516d4db2b3a85bda54bbbd8c; no voters:
I20260810 19:32:45.294353 6420 raft_consensus.cc:2781] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:45.294523 6420 raft_consensus.cc:482] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:45.294652 6420 raft_consensus.cc:3037] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:45.297065 6420 raft_consensus.cc:504] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.297715 6420 leader_election.cc:288] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 election: Requested vote from peers 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911), f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957)
I20260810 19:32:45.298244 6177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba760b8f4b74757ab2db6093ab02831" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7006b47b0979434d9683a6f340627ee4"
I20260810 19:32:45.298264 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba760b8f4b74757ab2db6093ab02831" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c"
I20260810 19:32:45.298527 6177 raft_consensus.cc:3037] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:45.298532 6331 raft_consensus.cc:3037] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:45.300905 6331 raft_consensus.cc:2454] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 1.
I20260810 19:32:45.300918 6177 raft_consensus.cc:2454] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 1.
I20260810 19:32:45.301371 5951 leader_election.cc:302] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b03dfb8ab4034d3483dd6db7b65bffe6, f7f2920c516d4db2b3a85bda54bbbd8c; no voters:
I20260810 19:32:45.301692 6420 raft_consensus.cc:2781] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:45.302153 6420 raft_consensus.cc:686] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 LEADER]: Becoming Leader. State: Replica: b03dfb8ab4034d3483dd6db7b65bffe6, State: Running, Role: LEADER
I20260810 19:32:45.302843 6420 consensus_queue.cc:237] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } }
I20260810 19:32:45.307543 5022 catalog_manager.cc:5495] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:45.339561 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.334035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.29.233.129:39067)
0810 19:32:45.334256 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.334262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.334324 (+ 62us) server_negotiation.cc:407] Connection header received
0810 19:32:45.334439 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.334443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.334525 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.334691 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:45.335106 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.335869 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.337286 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.337719 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.337860 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.338415 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:45.338935 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:45.339147 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.339204 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.339286 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:32:45.348114 6177 raft_consensus.cc:1261] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Refusing update from remote peer b03dfb8ab4034d3483dd6db7b65bffe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:45.348109 6331 raft_consensus.cc:1261] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Refusing update from remote peer b03dfb8ab4034d3483dd6db7b65bffe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:45.349161 6424 consensus_queue.cc:1035] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:45.349537 6420 consensus_queue.cc:1035] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:45.368606 6428 mvcc.cc:205] Tried to move back new op lower bound from 7317054936455606272 to 7317054936284446720. Current Snapshot: MvccSnapshot[applied={T|T < 7317054936455606272}]
I20260810 19:32:45.373098 6429 mvcc.cc:205] Tried to move back new op lower bound from 7317054936455606272 to 7317054936284446720. Current Snapshot: MvccSnapshot[applied={T|T < 7317054936455606272 or (T in {7317054936455606272})}]
I20260810 19:32:45.378897 6430 mvcc.cc:205] Tried to move back new op lower bound from 7317054936455606272 to 7317054936284446720. Current Snapshot: MvccSnapshot[applied={T|T < 7317054936455606272}]
I20260810 19:32:45.388545 5023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:45.388825 5023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:45.395290 5023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfd2beedb5444b96b423800774185423: Sending DeleteTablet for 3 replicas of tablet 9ba760b8f4b74757ab2db6093ab02831
I20260810 19:32:45.396462 5996 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba760b8f4b74757ab2db6093ab02831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:45 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:47834
I20260810 19:32:45.396673 6309 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba760b8f4b74757ab2db6093ab02831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:45 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:33098
I20260810 19:32:45.397621 6156 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba760b8f4b74757ab2db6093ab02831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:45 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:33148
I20260810 19:32:45.403846 6440 tablet_replica.cc:330] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: stopping tablet replica
I20260810 19:32:45.404922 6440 raft_consensus.cc:2229] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:45.406036 6440 raft_consensus.cc:2258] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:45.409518 6441 tablet_replica.cc:330] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: stopping tablet replica
I20260810 19:32:45.410105 6441 raft_consensus.cc:2229] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:45.410673 6441 raft_consensus.cc:2258] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:45.412806 6441 ts_tablet_manager.cc:1905] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:45.418672 6440 ts_tablet_manager.cc:1905] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:45.420655 6441 ts_tablet_manager.cc:1918] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:45.420900 6441 log.cc:1196] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/9ba760b8f4b74757ab2db6093ab02831
I20260810 19:32:45.421561 6441 ts_tablet_manager.cc:1939] T 9ba760b8f4b74757ab2db6093ab02831 P 7006b47b0979434d9683a6f340627ee4: Deleting consensus metadata
I20260810 19:32:45.423652 5008 catalog_manager.cc:4841] TS 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911): tablet 9ba760b8f4b74757ab2db6093ab02831 (table test-table [id=a677c521dcfd47baa76a8e7a205ffd12]) successfully deleted
I20260810 19:32:45.425448 6440 ts_tablet_manager.cc:1918] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:45.425654 6440 log.cc:1196] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/9ba760b8f4b74757ab2db6093ab02831
I20260810 19:32:45.426117 6440 ts_tablet_manager.cc:1939] T 9ba760b8f4b74757ab2db6093ab02831 P f7f2920c516d4db2b3a85bda54bbbd8c: Deleting consensus metadata
I20260810 19:32:45.427906 5009 catalog_manager.cc:4841] TS f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957): tablet 9ba760b8f4b74757ab2db6093ab02831 (table test-table [id=a677c521dcfd47baa76a8e7a205ffd12]) successfully deleted
I20260810 19:32:45.429306 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 4989
I20260810 19:32:45.430644 6439 tablet_replica.cc:330] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: stopping tablet replica
I20260810 19:32:45.431155 6439 raft_consensus.cc:2229] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:45.431932 6439 raft_consensus.cc:2258] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:45.433646 5054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:45.433808 6439 ts_tablet_manager.cc:1905] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:45.442127 6439 ts_tablet_manager.cc:1918] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:45.442378 6439 log.cc:1196] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/9ba760b8f4b74757ab2db6093ab02831
I20260810 19:32:45.443073 6439 ts_tablet_manager.cc:1939] T 9ba760b8f4b74757ab2db6093ab02831 P b03dfb8ab4034d3483dd6db7b65bffe6: Deleting consensus metadata
I20260810 19:32:45.451182 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.401662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.29.233.189:34267)
0810 19:32:45.401843 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.401849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.426700 (+ 24851us) server_negotiation.cc:407] Connection header received
0810 19:32:45.429889 (+ 3189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.429897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.430018 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.430098 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:45.435509 (+ 5411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.436654 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.438922 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.439635 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.439742 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.443633 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:45.443641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:45.443646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:45.443693 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:45.447582 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.448531 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.448537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.448541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.448706 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.449176 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.449182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.449185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.449578 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:45.449836 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.450450 (+ 614us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.450763 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 19:32:45.468722 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.427324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.29.233.188:40533)
0810 19:32:45.427527 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.427533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.451252 (+ 23719us) server_negotiation.cc:407] Connection header received
0810 19:32:45.451904 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.451911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.451998 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.452082 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:45.454223 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.455458 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.457418 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.458046 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.458155 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.461802 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:45.461810 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:45.461816 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:45.461859 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:45.465516 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.466362 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.466369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.466373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.466550 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.467041 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.467047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.467050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.467463 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:45.467714 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.468062 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.468292 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 19:32:45.569756 5010 catalog_manager.cc:4841] TS b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129:39067): tablet 9ba760b8f4b74757ab2db6093ab02831 (table test-table [id=a677c521dcfd47baa76a8e7a205ffd12]) successfully deleted
I20260810 19:32:45.570636 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 4989
I20260810 19:32:45.644239 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.610609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.29.233.188:40533)
0810 19:32:45.610814 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.610821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.621891 (+ 11070us) server_negotiation.cc:407] Connection header received
0810 19:32:45.622486 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.622493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.622580 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.622659 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:45.624998 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.626250 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.628589 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.629320 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.629437 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.637199 (+ 7762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:45.637206 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:45.637211 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:45.637253 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:45.640733 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.641550 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.641559 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.641563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.641742 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.642219 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.642224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.642227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.642660 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:45.642905 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.643583 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.643861 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 19:32:45.647339 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:45.629744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.29.233.189:34267)
0810 19:32:45.629952 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:45.629958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:45.629977 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:32:45.630037 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:45.630043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:45.630126 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:45.630616 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:45.632440 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.633611 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.635584 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:45.636279 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:45.636432 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:45.639554 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:45.639560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:45.639563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:45.639660 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:45.643263 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.644516 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.644520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.644522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.644647 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:45.645683 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:45.645688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:45.645690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:45.646072 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:45.646343 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:45.646753 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 19:32:45.647017 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 19:32:46.364892 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.364211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.131:44545)
0810 19:32:46.364606 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.364718 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":206,"threads_started":1}
W20260810 19:32:46.365402 6378 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:46487 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.381606 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.380804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:48559)
0810 19:32:46.381290 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.381405 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":259,"threads_started":1}
W20260810 19:32:46.382030 6066 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:46487 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.386193 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.385445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.130:38989)
0810 19:32:46.385922 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.386031 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":267,"threads_started":1}
W20260810 19:32:46.386588 6229 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:46487 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.895429 6454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cfd2beedb5444b96b423800774185423)
I20260810 19:32:46.895758 6454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } }
I20260810 19:32:46.897132 6454 leader_election.cc:288] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cfd2beedb5444b96b423800774185423 (127.29.233.190:46487), 519617e664494d928d11e37fb078ba8a (127.29.233.188:40533)
I20260810 19:32:46.898422 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.897268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60140)
0810 19:32:46.897859 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.898088 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":461,"threads_started":1}
W20260810 19:32:46.898763 5656 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.899331 6458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cfd2beedb5444b96b423800774185423)
I20260810 19:32:46.899715 6458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } }
I20260810 19:32:46.900470 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.900069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60148)
0810 19:32:46.900253 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.900333 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260810 19:32:46.900703 5656 leader_election.cc:334] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cfd2beedb5444b96b423800774185423 (127.29.233.190:46487): Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.901265 6458 leader_election.cc:288] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cfd2beedb5444b96b423800774185423 (127.29.233.190:46487), 7e35c1c720704f8e9fd1433d2bfc6fb1 (127.29.233.189:34267)
I20260810 19:32:46.902376 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.901304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60164)
0810 19:32:46.901773 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.901989 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":237,"threads_started":1}
W20260810 19:32:46.904536 5857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.905582 6457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.897601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.29.233.188:40533)
0810 19:32:46.898345 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:46.898352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:46.898372 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:46.898481 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:46.898486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:46.898566 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:46.898677 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:46.899562 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.900830 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.903248 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.904415 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.904579 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.904670 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:46.905112 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 19:32:46.905271 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":467,"threads_started":1}
I20260810 19:32:46.906270 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.897398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:40533 (local address 127.0.0.1:34746)
0810 19:32:46.898043 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.898082 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:46.898231 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:46.898729 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:46.898737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:46.898782 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:46.899281 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.899302 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.901007 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.901018 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.903025 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.903046 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.904589 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.904596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.904872 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.904917 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 19:32:46.905773 (+ 856us) client_negotiation.cc:240] Negotiation successful
0810 19:32:46.905912 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":25,"thread_start_us":308,"threads_started":1}
I20260810 19:32:46.906701 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.905844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60170)
0810 19:32:46.906096 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.906279 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 19:32:46.906962 5857 leader_election.cc:334] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cfd2beedb5444b96b423800774185423 (127.29.233.190:46487): Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.907094 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "519617e664494d928d11e37fb078ba8a" is_pre_election: true
I20260810 19:32:46.907462 5883 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e35c1c720704f8e9fd1433d2bfc6fb1 in term 1.
I20260810 19:32:46.907951 5653 leader_election.cc:302] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 519617e664494d928d11e37fb078ba8a, 7e35c1c720704f8e9fd1433d2bfc6fb1; no voters: cfd2beedb5444b96b423800774185423
I20260810 19:32:46.908337 6454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 19:32:46.908492 6454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cfd2beedb5444b96b423800774185423)
I20260810 19:32:46.908631 6454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 1 FOLLOWER]: Advancing to term 2
I20260810 19:32:46.910053 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.901938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:34267 (local address 127.0.0.1:42598)
0810 19:32:46.902674 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.902695 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:46.902828 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:46.903817 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:46.903822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:46.903851 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:46.904200 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.904213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.905839 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.905845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.907874 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.907886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.909291 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.909298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.909484 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.909508 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:32:46.909634 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 19:32:46.909770 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":2277,"threads_started":1}
I20260810 19:32:46.910435 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.902004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.29.233.189:34267)
0810 19:32:46.903459 (+ 1455us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:46.903467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:46.903496 (+ 29us) server_negotiation.cc:407] Connection header received
0810 19:32:46.903564 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:46.903573 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:46.903663 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:46.903783 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:46.904422 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.905635 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.908028 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.909113 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.909309 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.909388 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:46.909658 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 19:32:46.909800 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1287,"thread_start_us":1107,"threads_started":1}
I20260810 19:32:46.911068 5679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "519617e664494d928d11e37fb078ba8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" is_pre_election: true
I20260810 19:32:46.911464 6454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } }
I20260810 19:32:46.911876 5679 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 519617e664494d928d11e37fb078ba8a in current term 2: Already voted for candidate 7e35c1c720704f8e9fd1433d2bfc6fb1 in this term.
I20260810 19:32:46.912222 6454 leader_election.cc:288] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [CANDIDATE]: Term 2 election: Requested vote from peers cfd2beedb5444b96b423800774185423 (127.29.233.190:46487), 519617e664494d928d11e37fb078ba8a (127.29.233.188:40533)
I20260810 19:32:46.912616 5857 leader_election.cc:302] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 519617e664494d928d11e37fb078ba8a; no voters: 7e35c1c720704f8e9fd1433d2bfc6fb1, cfd2beedb5444b96b423800774185423
I20260810 19:32:46.912729 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.912295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60174)
0810 19:32:46.912498 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.912587 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260810 19:32:46.912705 5883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "519617e664494d928d11e37fb078ba8a"
I20260810 19:32:46.912933 5883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 1 FOLLOWER]: Advancing to term 2
I20260810 19:32:46.913761 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.913395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60178)
0810 19:32:46.913560 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.913651 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 19:32:46.913951 5656 leader_election.cc:334] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cfd2beedb5444b96b423800774185423 (127.29.233.190:46487): Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
I20260810 19:32:46.916172 5883 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e35c1c720704f8e9fd1433d2bfc6fb1 in term 2.
I20260810 19:32:46.916456 6458 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 19:32:46.916602 5653 leader_election.cc:302] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 519617e664494d928d11e37fb078ba8a, 7e35c1c720704f8e9fd1433d2bfc6fb1; no voters: cfd2beedb5444b96b423800774185423
I20260810 19:32:46.917029 6454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 19:32:46.917546 6454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [term 2 LEADER]: Becoming Leader. State: Replica: 7e35c1c720704f8e9fd1433d2bfc6fb1, State: Running, Role: LEADER
I20260810 19:32:46.917915 6454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } }
I20260810 19:32:46.920670 6465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e35c1c720704f8e9fd1433d2bfc6fb1. Latest consensus state: current_term: 2 leader_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:46.920964 6465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:46.921550 6467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:46.925148 6467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:46.925973 6467 catalog_manager.cc:1259] Loaded cluster ID: a9a7add4949e428395feb7252859a6ac
I20260810 19:32:46.926049 6467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:46.927309 6467 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:46.928215 6467 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Loaded TSK: 0
I20260810 19:32:46.929069 6467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:32:46.944258 5668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:46.951475 5668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:46.955792 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.955360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60196)
0810 19:32:46.955532 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.955630 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 19:32:46.955921 5883 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [term 2 FOLLOWER]: Refusing update from remote peer 7e35c1c720704f8e9fd1433d2bfc6fb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 19:32:46.956833 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.956462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60212)
0810 19:32:46.956610 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.956688 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 19:32:46.956735 6465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 19:32:46.957024 5656 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 -> Peer cfd2beedb5444b96b423800774185423 (127.29.233.190:46487): Couldn't send request to peer cfd2beedb5444b96b423800774185423. Status: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 19:32:46.959568 6458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e35c1c720704f8e9fd1433d2bfc6fb1. Latest consensus state: current_term: 2 leader_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:46.959846 6458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:32:46.961440 6465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:46.961730 6465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1 [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:46.961944 6469 mvcc.cc:205] Tried to move back new op lower bound from 7317054943044374528 to 7317054942900015104. Current Snapshot: MvccSnapshot[applied={T|T < 7317054943044374528}]
I20260810 19:32:46.962910 6458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cfd2beedb5444b96b423800774185423" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46487 } } peers { permanent_uuid: "7e35c1c720704f8e9fd1433d2bfc6fb1" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 34267 } attrs { promote: false } } peers { permanent_uuid: "519617e664494d928d11e37fb078ba8a" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 40533 } attrs { promote: false } } }
I20260810 19:32:46.963188 6458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519617e664494d928d11e37fb078ba8a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:32:46.985337 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.970272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45957 (local address 127.0.0.1:33102)
0810 19:32:46.970437 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.970466 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:46.970561 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:46.973007 (+ 2446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:46.973014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:46.973067 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:46.973342 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.973371 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.976714 (+ 3343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.976723 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.981673 (+ 4950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.981691 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.984402 (+ 2711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.984409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.984640 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.984675 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 19:32:46.984762 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 19:32:46.984915 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":1}
I20260810 19:32:46.986258 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.970633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.29.233.131:45957)
0810 19:32:46.972380 (+ 1747us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:46.972389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:46.972415 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:32:46.972509 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:46.972517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:46.972624 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:46.972751 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:46.973568 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.976502 (+ 2934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.981882 (+ 5380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.984056 (+ 2174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.984223 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.984300 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:46.985793 (+ 1493us) server_negotiation.cc:299] Negotiation successful
0810 19:32:46.985948 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1570,"thread_start_us":1451,"threads_started":1}
I20260810 19:32:46.987118 6309 tablet_service.cc:1467] Processing CreateTablet for tablet 754e994bb5f745d0a465f947d7b4f8c8 (DEFAULT_TABLE table=test-table [id=d61a9e6f1c484b10bd08d54cca7a6319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:46.987336 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.969739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39067 (local address 127.0.0.1:47864)
0810 19:32:46.969929 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.969953 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:46.970060 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:46.982379 (+ 12319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:46.982387 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:46.982425 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:46.982772 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.982781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.983933 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.983939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.985258 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.985269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.986659 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.986665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.986861 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.986894 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 19:32:46.986980 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 19:32:46.987100 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 19:32:46.987663 6309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754e994bb5f745d0a465f947d7b4f8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:46.988233 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.970307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.29.233.129:39067)
0810 19:32:46.981959 (+ 11652us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:46.981965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:46.981985 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:46.982055 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:46.982060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:46.982142 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:46.982321 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:46.982932 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.983786 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.985454 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.986388 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.987695 (+ 1307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.987764 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:46.987817 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 19:32:46.987943 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11493,"thread_start_us":11794,"threads_started":1}
I20260810 19:32:46.989060 5996 tablet_service.cc:1467] Processing CreateTablet for tablet 754e994bb5f745d0a465f947d7b4f8c8 (DEFAULT_TABLE table=test-table [id=d61a9e6f1c484b10bd08d54cca7a6319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:46.989488 5996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754e994bb5f745d0a465f947d7b4f8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:46.989531 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.970670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:42911 (local address 127.0.0.1:33160)
0810 19:32:46.981833 (+ 11163us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:46.981864 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:46.982021 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:46.983266 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:46.983271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:46.983294 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:46.983667 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.983680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.985062 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.985067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.987024 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:46.987038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.988821 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.988826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:46.988940 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.988963 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:32:46.989092 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 19:32:46.989213 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10996,"thread_start_us":10749,"threads_started":1}
I20260810 19:32:46.989555 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:46.971034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.29.233.130:42911)
0810 19:32:46.982814 (+ 11780us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:46.982821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:46.982841 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:46.982912 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:46.982917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:46.983000 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:46.983105 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:46.983852 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.984902 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.987193 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:46.988437 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:46.988604 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:46.988675 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:46.989120 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 19:32:46.989267 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11651,"thread_start_us":11540,"threads_started":1}
I20260810 19:32:46.990427 6156 tablet_service.cc:1467] Processing CreateTablet for tablet 754e994bb5f745d0a465f947d7b4f8c8 (DEFAULT_TABLE table=test-table [id=d61a9e6f1c484b10bd08d54cca7a6319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:46.990857 6156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754e994bb5f745d0a465f947d7b4f8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:46.994962 6479 tablet_bootstrap.cc:492] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: Bootstrap starting.
I20260810 19:32:46.996220 6480 tablet_bootstrap.cc:492] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: Bootstrap starting.
I20260810 19:32:46.996712 6479 tablet_bootstrap.cc:654] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:46.996955 6481 tablet_bootstrap.cc:492] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: Bootstrap starting.
I20260810 19:32:46.998054 6480 tablet_bootstrap.cc:654] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:46.998832 6481 tablet_bootstrap.cc:654] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:46.999078 6479 tablet_bootstrap.cc:492] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: No bootstrap required, opened a new log
I20260810 19:32:46.999352 6479 ts_tablet_manager.cc:1397] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260810 19:32:47.000787 6480 tablet_bootstrap.cc:492] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: No bootstrap required, opened a new log
I20260810 19:32:47.000936 6480 ts_tablet_manager.cc:1397] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 19:32:47.000543 6479 raft_consensus.cc:348] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.001138 6481 tablet_bootstrap.cc:492] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: No bootstrap required, opened a new log
I20260810 19:32:47.001232 6479 raft_consensus.cc:374] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:47.001286 6481 ts_tablet_manager.cc:1397] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 19:32:47.001344 6479 raft_consensus.cc:729] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f2920c516d4db2b3a85bda54bbbd8c, State: Initialized, Role: FOLLOWER
I20260810 19:32:47.001602 6479 consensus_queue.cc:260] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.001856 6480 raft_consensus.cc:348] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.002169 6480 raft_consensus.cc:374] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:47.002064 6481 raft_consensus.cc:348] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.002368 6481 raft_consensus.cc:374] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:47.002358 6480 raft_consensus.cc:729] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03dfb8ab4034d3483dd6db7b65bffe6, State: Initialized, Role: FOLLOWER
I20260810 19:32:47.002467 6479 ts_tablet_manager.cc:1428] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:47.002512 6481 raft_consensus.cc:729] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7006b47b0979434d9683a6f340627ee4, State: Initialized, Role: FOLLOWER
I20260810 19:32:47.002820 6481 consensus_queue.cc:260] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.003796 6481 ts_tablet_manager.cc:1428] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 19:32:47.005089 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.004018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.130:60185)
0810 19:32:47.004634 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.004805 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"threads_started":1}
I20260810 19:32:47.007776 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.006388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.130:42829)
0810 19:32:47.007578 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.007664 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1056}
I20260810 19:32:47.009095 6230 heartbeater.cc:503] Master 127.29.233.189:34267 was elected leader, sending a full tablet report...
I20260810 19:32:47.002662 6480 consensus_queue.cc:260] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.012202 6480 ts_tablet_manager.cc:1428] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20260810 19:32:47.012929 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.011759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:39893)
0810 19:32:47.012388 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.012595 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":318,"threads_started":1}
I20260810 19:32:47.013931 6067 heartbeater.cc:503] Master 127.29.233.189:34267 was elected leader, sending a full tablet report...
I20260810 19:32:47.014637 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.014035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:49789)
0810 19:32:47.014324 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.014432 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260810 19:32:47.017791 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.003124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.131:33729)
0810 19:32:47.017469 (+ 14345us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.017613 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14240,"threads_started":1}
I20260810 19:32:47.019158 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.018726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.131:38071)
0810 19:32:47.018908 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.019014 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 19:32:47.035115 6489 raft_consensus.cc:482] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:47.035379 6489 raft_consensus.cc:504] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.036140 6489 leader_election.cc:288] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957), 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911)
I20260810 19:32:47.036659 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754e994bb5f745d0a465f947d7b4f8c8" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" is_pre_election: true
I20260810 19:32:47.036659 6177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754e994bb5f745d0a465f947d7b4f8c8" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7006b47b0979434d9683a6f340627ee4" is_pre_election: true
I20260810 19:32:47.036890 6331 raft_consensus.cc:2454] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 0.
I20260810 19:32:47.036926 6177 raft_consensus.cc:2454] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 0.
I20260810 19:32:47.037395 5951 leader_election.cc:302] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b03dfb8ab4034d3483dd6db7b65bffe6, f7f2920c516d4db2b3a85bda54bbbd8c; no voters:
I20260810 19:32:47.037705 6489 raft_consensus.cc:2781] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:47.037856 6489 raft_consensus.cc:482] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:47.037979 6489 raft_consensus.cc:3037] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:47.040536 6489 raft_consensus.cc:504] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.041180 6489 leader_election.cc:288] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 election: Requested vote from peers f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957), 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911)
I20260810 19:32:47.041664 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754e994bb5f745d0a465f947d7b4f8c8" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c"
I20260810 19:32:47.041831 6177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754e994bb5f745d0a465f947d7b4f8c8" candidate_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7006b47b0979434d9683a6f340627ee4"
I20260810 19:32:47.041916 6331 raft_consensus.cc:3037] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:47.042060 6177 raft_consensus.cc:3037] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:47.045045 6177 raft_consensus.cc:2454] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 1.
I20260810 19:32:47.045033 6331 raft_consensus.cc:2454] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b03dfb8ab4034d3483dd6db7b65bffe6 in term 1.
I20260810 19:32:47.045517 5948 leader_election.cc:302] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7006b47b0979434d9683a6f340627ee4, b03dfb8ab4034d3483dd6db7b65bffe6; no voters:
I20260810 19:32:47.045929 6489 raft_consensus.cc:2781] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:47.046393 6489 raft_consensus.cc:686] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 LEADER]: Becoming Leader. State: Replica: b03dfb8ab4034d3483dd6db7b65bffe6, State: Running, Role: LEADER
I20260810 19:32:47.046975 6489 consensus_queue.cc:237] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } }
I20260810 19:32:47.047510 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.047048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:57123)
0810 19:32:47.047224 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.047334 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 19:32:47.049892 5669 catalog_manager.cc:5495] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b03dfb8ab4034d3483dd6db7b65bffe6" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:47.080166 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.075481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47876 (local address 127.29.233.129:39067)
0810 19:32:47.075698 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:47.075703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:47.075723 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:47.075898 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:47.075901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:47.075968 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:47.076044 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:47.076398 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.077092 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.078437 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.078877 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.078971 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:47.079516 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:47.079627 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:47.079697 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:47.079811 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 19:32:47.079872 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:32:47.083410 6331 raft_consensus.cc:1261] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Refusing update from remote peer b03dfb8ab4034d3483dd6db7b65bffe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:47.083417 6177 raft_consensus.cc:1261] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Refusing update from remote peer b03dfb8ab4034d3483dd6db7b65bffe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:47.084123 6492 consensus_queue.cc:1035] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7006b47b0979434d9683a6f340627ee4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 42911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:47.084465 6489 consensus_queue.cc:1035] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f2920c516d4db2b3a85bda54bbbd8c" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:47.086127 6379 heartbeater.cc:503] Master 127.29.233.189:34267 was elected leader, sending a full tablet report...
I20260810 19:32:47.087375 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.086736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.130:54233)
0810 19:32:47.086979 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.087115 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 19:32:47.088109 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.086736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.131:43803)
0810 19:32:47.087709 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.087832 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":818}
I20260810 19:32:47.096488 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.096029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.130:38469)
0810 19:32:47.096223 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.096310 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 19:32:47.097987 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.097400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.131:60897)
0810 19:32:47.097671 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.097783 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260810 19:32:47.104075 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.103571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:37393)
0810 19:32:47.103794 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.103892 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 19:32:47.105441 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.104936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.29.233.129:48731)
0810 19:32:47.105125 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.105215 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 19:32:47.110656 5668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:47.110952 5668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:47.116442 5668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e35c1c720704f8e9fd1433d2bfc6fb1: Sending DeleteTablet for 3 replicas of tablet 754e994bb5f745d0a465f947d7b4f8c8
I20260810 19:32:47.117689 5996 tablet_service.cc:1514] Processing DeleteTablet for tablet 754e994bb5f745d0a465f947d7b4f8c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:47 UTC) from {username='oryx', principal='oryx/127.29.233.189@KRBTEST.COM'} at 127.0.0.1:47864
I20260810 19:32:47.117921 6309 tablet_service.cc:1514] Processing DeleteTablet for tablet 754e994bb5f745d0a465f947d7b4f8c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:47 UTC) from {username='oryx', principal='oryx/127.29.233.189@KRBTEST.COM'} at 127.0.0.1:33102
I20260810 19:32:47.118180 6156 tablet_service.cc:1514] Processing DeleteTablet for tablet 754e994bb5f745d0a465f947d7b4f8c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:47 UTC) from {username='oryx', principal='oryx/127.29.233.189@KRBTEST.COM'} at 127.0.0.1:33160
I20260810 19:32:47.118501 6505 tablet_replica.cc:330] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: stopping tablet replica
I20260810 19:32:47.118736 6505 raft_consensus.cc:2229] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:47.119243 6505 raft_consensus.cc:2258] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:47.120589 6505 ts_tablet_manager.cc:1905] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:47.122781 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 5858
I20260810 19:32:47.129887 6506 tablet_replica.cc:330] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: stopping tablet replica
I20260810 19:32:47.130196 6506 raft_consensus.cc:2229] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:47.130283 6505 ts_tablet_manager.cc:1918] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:47.130461 6505 log.cc:1196] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/754e994bb5f745d0a465f947d7b4f8c8
I20260810 19:32:47.130733 6506 raft_consensus.cc:2258] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:47.131018 6505 ts_tablet_manager.cc:1939] T 754e994bb5f745d0a465f947d7b4f8c8 P b03dfb8ab4034d3483dd6db7b65bffe6: Deleting consensus metadata
I20260810 19:32:47.131467 6507 tablet_replica.cc:330] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: stopping tablet replica
I20260810 19:32:47.131665 6507 raft_consensus.cc:2229] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:47.131966 6507 raft_consensus.cc:2258] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:47.132050 6506 ts_tablet_manager.cc:1905] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:47.132676 5656 catalog_manager.cc:4841] TS b03dfb8ab4034d3483dd6db7b65bffe6 (127.29.233.129:39067): tablet 754e994bb5f745d0a465f947d7b4f8c8 (table test-table [id=d61a9e6f1c484b10bd08d54cca7a6319]) successfully deleted
I20260810 19:32:47.133139 6507 ts_tablet_manager.cc:1905] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:47.137674 6507 ts_tablet_manager.cc:1918] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:47.137841 6507 log.cc:1196] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/754e994bb5f745d0a465f947d7b4f8c8
I20260810 19:32:47.138275 6507 ts_tablet_manager.cc:1939] T 754e994bb5f745d0a465f947d7b4f8c8 P 7006b47b0979434d9683a6f340627ee4: Deleting consensus metadata
I20260810 19:32:47.139151 6506 ts_tablet_manager.cc:1918] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:47.139298 6506 log.cc:1196] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/754e994bb5f745d0a465f947d7b4f8c8
I20260810 19:32:47.139585 5654 catalog_manager.cc:4841] TS 7006b47b0979434d9683a6f340627ee4 (127.29.233.130:42911): tablet 754e994bb5f745d0a465f947d7b4f8c8 (table test-table [id=d61a9e6f1c484b10bd08d54cca7a6319]) successfully deleted
I20260810 19:32:47.139804 6506 ts_tablet_manager.cc:1939] T 754e994bb5f745d0a465f947d7b4f8c8 P f7f2920c516d4db2b3a85bda54bbbd8c: Deleting consensus metadata
I20260810 19:32:47.140830 5655 catalog_manager.cc:4841] TS f7f2920c516d4db2b3a85bda54bbbd8c (127.29.233.131:45957): tablet 754e994bb5f745d0a465f947d7b4f8c8 (table test-table [id=d61a9e6f1c484b10bd08d54cca7a6319]) successfully deleted
I20260810 19:32:47.151908 6475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.124163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.29.233.129:39067)
0810 19:32:47.124385 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:47.124394 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:47.138144 (+ 13750us) server_negotiation.cc:407] Connection header received
0810 19:32:47.138385 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:47.138391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:47.138475 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:47.138585 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:47.140752 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.141614 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.142989 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.143391 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.143486 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:47.146019 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:47.146043 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:47.146061 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:47.146101 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:47.149115 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:47.149882 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:47.149888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:47.149890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:47.150016 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:47.150452 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:47.150455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:47.150457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:47.150813 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:47.151132 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:47.151381 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 19:32:47.151545 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:32:47.152560 6060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:47.287020 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 5858
I20260810 19:32:47.314761 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 6078
I20260810 19:32:47.327839 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.315605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33172 (local address 127.29.233.130:42911)
0810 19:32:47.315826 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:47.315831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:47.315849 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:32:47.315926 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:47.315930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:47.315992 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:47.316089 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:47.317409 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.318109 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.319423 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.319822 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.319918 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:47.322409 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:47.322426 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:47.322444 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:47.322478 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:47.325257 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:47.325934 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:47.325941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:47.325943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:47.326061 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:47.326449 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:47.326452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:47.326454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:47.326785 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:47.327026 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:47.327251 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 19:32:47.327482 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:32:47.328617 6222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:47.435011 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 6078
I20260810 19:32:47.456945 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 6241
I20260810 19:32:47.458791 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.458325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60228)
0810 19:32:47.458486 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.458600 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 19:32:47.460187 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.459763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46487 (local address 127.0.0.1:60232)
0810 19:32:47.459949 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:47.460042 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:46487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 19:32:47.471936 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:47.457847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.29.233.131:45957)
0810 19:32:47.458080 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:47.458088 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:47.458114 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:32:47.458326 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:47.458332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:47.458443 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:47.458557 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:47.460363 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.461096 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.462564 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:47.462990 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:47.463143 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:47.465824 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:47.465853 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:47.465871 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:47.465905 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:47.469019 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:47.469720 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:47.469732 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:47.469736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:47.469878 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:47.470283 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:47.470288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:47.470291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:47.470721 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:47.471026 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:47.471354 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 19:32:47.471603 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:32:47.472633 6372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:47.567194 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 6241
I20260810 19:32:47.588609 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 5519
I20260810 19:32:47.589612 5699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:47.716549 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 5519
I20260810 19:32:47.743104 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 5629
I20260810 19:32:47.744524 5905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:47.863518 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 5629
2026-08-10T19:32:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10002 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:47 dist-test-slave-tm1g krb5kdc[6517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:47 dist-test-slave-tm1g krb5kdc[6517](info): set up 2 sockets
Aug 10 19:32:47 dist-test-slave-tm1g krb5kdc[6517](info): commencing operation
krb5kdc: starting...
W20260810 19:32:49.051450 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.007s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:49 dist-test-slave-tm1g krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab.
Entry for principal oryx/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab.
I20260810 19:32:49.331328 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39089
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab
--principal=oryx/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:39089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:49.679509 6531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:49.679816 6531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:49.680029 6531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:49.687422 6531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:49.687530 6531 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 19:32:49.687613 6531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:49.687713 6531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:49.687788 6531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:49.687872 6531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:49.698798 6531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:39089
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.190.keytab
--principal=oryx/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:49.700259 6531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:49.701655 6531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:49.714211 6536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:49.714445 6538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:49.714277 6540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:49.715441 6531 server_base.cc:1034] running on GCE node
I20260810 19:32:49.716265 6531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:49.717844 6531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:49.719108 6531 hybrid_clock.cc:648] HybridClock initialized: now 1786390369719026 us; error 50 us; skew 500 ppm
I20260810 19:32:49.719482 6531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:49 dist-test-slave-tm1g krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390369, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:49.722678 6531 init.cc:377] Logged in from keytab as oryx/127.29.233.190@KRBTEST.COM (short username oryx)
I20260810 19:32:49.724211 6531 webserver.cc:460] Webserver started at http://127.29.233.190:36025/ using document root <none> and password file <none>
I20260810 19:32:49.724957 6531 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:49.725092 6531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:49.725438 6531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:49.728619 6531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "5e19bcc5592f4dd6b42b511bf036fdef"
format_stamp: "Formatted at 2026-08-10 19:32:49 on dist-test-slave-tm1g"
server_key: "08333e7ba1a63e93d653936da3122344"
server_key_iv: "e9da5f5e4cc633b4b6fd9914a46dd1da"
server_key_version: "encryptionkey@0"
I20260810 19:32:49.729456 6531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "5e19bcc5592f4dd6b42b511bf036fdef"
format_stamp: "Formatted at 2026-08-10 19:32:49 on dist-test-slave-tm1g"
server_key: "08333e7ba1a63e93d653936da3122344"
server_key_iv: "e9da5f5e4cc633b4b6fd9914a46dd1da"
server_key_version: "encryptionkey@0"
I20260810 19:32:49.735220 6531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:32:49.738997 6547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:49.740418 6531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:49.740617 6531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "5e19bcc5592f4dd6b42b511bf036fdef"
format_stamp: "Formatted at 2026-08-10 19:32:49 on dist-test-slave-tm1g"
server_key: "08333e7ba1a63e93d653936da3122344"
server_key_iv: "e9da5f5e4cc633b4b6fd9914a46dd1da"
server_key_version: "encryptionkey@0"
I20260810 19:32:49.740856 6531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:49.757946 6531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:49.762570 6531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:49.762876 6531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:49.785622 6531 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:39089
I20260810 19:32:49.785661 6611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:39089 every 8 connection(s)
I20260810 19:32:49.787081 6531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:49.788367 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 6531
I20260810 19:32:49.788589 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:49.789477 30630 external_mini_cluster.cc:1428] Setting key 221914518b8c14b9fc79b9478938096e
I20260810 19:32:49.792145 6613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 19:32:49 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:49.805985 6613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Bootstrap starting.
I20260810 19:32:49.810261 6613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:49.811784 6613 log.cc:824] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:49.815485 6613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: No bootstrap required, opened a new log
I20260810 19:32:49.815693 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:49.791698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.29.233.190:39089)
0810 19:32:49.792535 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:49.792547 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:49.792587 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:32:49.794216 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:49.794236 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:49.794698 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:49.795056 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:49.796945 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:49.798837 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:49.800601 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:49.801393 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:49.801559 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:49.806688 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:49.806698 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:49.806717 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:49.806762 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:49.810947 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:49.811641 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:49.811649 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:49.811657 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:49.811987 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:49.812382 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:49.812387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:49.812390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:49.813101 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:49.813383 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:49.813883 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 19:32:49.814362 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":408,"threads_started":1}
I20260810 19:32:49.824354 6613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39089 } }
I20260810 19:32:49.824708 6613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:49.824786 6613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e19bcc5592f4dd6b42b511bf036fdef, State: Initialized, Role: FOLLOWER
I20260810 19:32:49.825274 6613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39089 } }
I20260810 19:32:49.825526 6613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:49.825621 6613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:49.825763 6613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:49.827955 6613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39089 } }
I20260810 19:32:49.828466 6613 leader_election.cc:302] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e19bcc5592f4dd6b42b511bf036fdef; no voters:
I20260810 19:32:49.829003 6613 leader_election.cc:288] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:49.829152 6618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:49.830381 6618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [term 1 LEADER]: Becoming Leader. State: Replica: 5e19bcc5592f4dd6b42b511bf036fdef, State: Running, Role: LEADER
I20260810 19:32:49.831146 6618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39089 } }
I20260810 19:32:49.831462 6613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:49.838415 6620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e19bcc5592f4dd6b42b511bf036fdef. Latest consensus state: current_term: 1 leader_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39089 } } }
I20260810 19:32:49.838402 6619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e19bcc5592f4dd6b42b511bf036fdef" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39089 } } }
I20260810 19:32:49.839083 6620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:49.839128 6619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:49.840060 6629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:49.845243 6629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:49.854452 6629 catalog_manager.cc:1347] Generated new cluster ID: 63ae1a687b2a4b6e936dbcc9d231a7c8
I20260810 19:32:49.854583 6629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:49.883085 6629 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:49.884452 6629 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:49.890537 6629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Generated new TSK 0
I20260810 19:32:49.891422 6629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab.
Entry for principal oryx/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab.
I20260810 19:32:49.963152 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39089
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab
--principal=oryx/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:50.307375 6642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:50.307693 6642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:50.307862 6642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:50.315750 6642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:50.315872 6642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:50.316053 6642 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:32:50.316188 6642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:50.326609 6642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab
--principal=oryx/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:39089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:50.328001 6642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:50.329442 6642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:50.343902 6651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:50.343982 6648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:50.344022 6649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:50.344867 6642 server_base.cc:1034] running on GCE node
I20260810 19:32:50.345518 6642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:50.346532 6642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:50.347788 6642 hybrid_clock.cc:648] HybridClock initialized: now 1786390370347722 us; error 40 us; skew 500 ppm
I20260810 19:32:50.348109 6642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:50 dist-test-slave-tm1g krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390370, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:50.351447 6642 init.cc:377] Logged in from keytab as oryx/127.29.233.129@KRBTEST.COM (short username oryx)
I20260810 19:32:50.352850 6642 webserver.cc:460] Webserver started at http://127.29.233.129:45199/ using document root <none> and password file <none>
I20260810 19:32:50.353525 6642 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:50.353610 6642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:50.353896 6642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:50.356895 6642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "fc11565f399d42b6b28391317d2aac65"
format_stamp: "Formatted at 2026-08-10 19:32:50 on dist-test-slave-tm1g"
server_key: "ce9d3c244a2ed86c9eaac91cbba63ccd"
server_key_iv: "14648bc563d539db9bf91469d7c6f47e"
server_key_version: "encryptionkey@0"
I20260810 19:32:50.357657 6642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "fc11565f399d42b6b28391317d2aac65"
format_stamp: "Formatted at 2026-08-10 19:32:50 on dist-test-slave-tm1g"
server_key: "ce9d3c244a2ed86c9eaac91cbba63ccd"
server_key_iv: "14648bc563d539db9bf91469d7c6f47e"
server_key_version: "encryptionkey@0"
I20260810 19:32:50.362726 6642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 19:32:50.366088 6658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:50.367480 6642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:50.367611 6642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "fc11565f399d42b6b28391317d2aac65"
format_stamp: "Formatted at 2026-08-10 19:32:50 on dist-test-slave-tm1g"
server_key: "ce9d3c244a2ed86c9eaac91cbba63ccd"
server_key_iv: "14648bc563d539db9bf91469d7c6f47e"
server_key_version: "encryptionkey@0"
I20260810 19:32:50.367799 6642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:50.386179 6642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:50.391131 6642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:50.391369 6642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:50.394129 6642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:50.394214 6642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:50.394399 6642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:50.394505 6642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:32:50 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390370, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:50.418596 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.394534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:49079 (local address 127.29.233.190:39089)
0810 19:32:50.395078 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:50.395084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:50.396307 (+ 1223us) server_negotiation.cc:407] Connection header received
0810 19:32:50.398417 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:50.398425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:50.398545 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:50.398705 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:50.401075 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.402579 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.405282 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.405894 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.406041 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.411357 (+ 5316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:50.411370 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:50.411374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:50.411412 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:50.414132 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.415334 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.415338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.415340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.415464 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.416118 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.416121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.416123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.416391 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:50.416511 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:50.417778 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0810 19:32:50.418073 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":239,"threads_started":1}
I20260810 19:32:50.420670 6668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.395219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39089 (local address 127.29.233.129:49079)
0810 19:32:50.396123 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:50.396173 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:50.398103 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:50.398995 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:50.399006 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:50.399856 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:50.400861 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.400881 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.402775 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.402783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.405105 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.405121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.406155 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.406162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.406526 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.407381 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:50.407436 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:50.411069 (+ 3633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:50.414296 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.414348 (+ 52us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.414374 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.415195 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.415586 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.415594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.415600 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.415986 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.416532 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:50.416545 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:50.417152 (+ 607us) client_negotiation.cc:769] Sending connection context
0810 19:32:50.417616 (+ 464us) client_negotiation.cc:240] Negotiation successful
0810 19:32:50.418425 (+ 809us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":383,"threads_started":1}
I20260810 19:32:50.447137 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.430000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:51719 (local address 127.29.233.190:39089)
0810 19:32:50.430223 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:50.430231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:50.431160 (+ 929us) server_negotiation.cc:407] Connection header received
0810 19:32:50.431494 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:50.431502 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:50.431602 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:50.431680 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:50.433527 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.434623 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.436798 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.437440 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.437594 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.440698 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:50.440703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:50.440706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:50.440738 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:50.443897 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.444940 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.444945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.444947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.445079 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.445588 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.445592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.445594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.445879 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:50.445996 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:50.446665 (+ 669us) server_negotiation.cc:299] Negotiation successful
0810 19:32:50.446879 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 19:32:50.447649 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.430027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39089 (local address 127.29.233.129:51719)
0810 19:32:50.431028 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:50.431056 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:50.431294 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:50.431955 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:50.431959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:50.432770 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:50.433357 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.433376 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.434779 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.434784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.436625 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.436642 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.437708 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.437718 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.437980 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.438920 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:50.438952 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:50.440445 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:50.444041 (+ 3596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.444049 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.444056 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.444797 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.445248 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.445256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.445262 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.445473 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.446002 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:50.446009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:50.446281 (+ 272us) client_negotiation.cc:769] Sending connection context
0810 19:32:50.446523 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 19:32:50.446966 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":588,"threads_started":1}
I20260810 19:32:50.456270 6642 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39421
I20260810 19:32:50.456295 6788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39421 every 8 connection(s)
I20260810 19:32:50.457567 6642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:50.459913 6789 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39089
I20260810 19:32:50.460158 6789 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:50.460857 6789 heartbeater.cc:511] Master 127.29.233.190:39089 requested a full tablet report, sending...
I20260810 19:32:50.462459 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 6642
I20260810 19:32:50.462680 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:50.463279 6565 ts_manager.cc:194] Registered new tserver with Master: fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421)
I20260810 19:32:50.463544 30630 external_mini_cluster.cc:1428] Setting key e4b7160e6004f246b480e336918c16e7
I20260810 19:32:50.465138 6565 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:49079
WARNING: no policy specified for oryx/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab.
Entry for principal oryx/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab.
I20260810 19:32:50.522249 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39089
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab
--principal=oryx/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:50.860306 6793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:50.860636 6793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:50.860816 6793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:50.868008 6793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:50.868182 6793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:50.868350 6793 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:32:50.868496 6793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:50.878482 6793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab
--principal=oryx/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:39089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:50.879824 6793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:50.881214 6793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:50.894949 6799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:50.894917 6798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:50.894920 6801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:50.895731 6793 server_base.cc:1034] running on GCE node
I20260810 19:32:50.896351 6793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:50.897240 6793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:50.898478 6793 hybrid_clock.cc:648] HybridClock initialized: now 1786390370898397 us; error 54 us; skew 500 ppm
I20260810 19:32:50.898797 6793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:50 dist-test-slave-tm1g krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390370, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:50.901959 6793 init.cc:377] Logged in from keytab as oryx/127.29.233.130@KRBTEST.COM (short username oryx)
I20260810 19:32:50.903424 6793 webserver.cc:460] Webserver started at http://127.29.233.130:45169/ using document root <none> and password file <none>
I20260810 19:32:50.904142 6793 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:50.904273 6793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:50.904603 6793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:50.907557 6793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "8af8509fd6b04d6299428eaaf0353dcf"
format_stamp: "Formatted at 2026-08-10 19:32:50 on dist-test-slave-tm1g"
server_key: "deba0f52480571e2fcef6b70c5bf2038"
server_key_iv: "55460f0c37586ce208fc2d10ca809406"
server_key_version: "encryptionkey@0"
I20260810 19:32:50.908362 6793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "8af8509fd6b04d6299428eaaf0353dcf"
format_stamp: "Formatted at 2026-08-10 19:32:50 on dist-test-slave-tm1g"
server_key: "deba0f52480571e2fcef6b70c5bf2038"
server_key_iv: "55460f0c37586ce208fc2d10ca809406"
server_key_version: "encryptionkey@0"
I20260810 19:32:50.913550 6793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:32:50.916929 6809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:50.918246 6793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:50.918447 6793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "8af8509fd6b04d6299428eaaf0353dcf"
format_stamp: "Formatted at 2026-08-10 19:32:50 on dist-test-slave-tm1g"
server_key: "deba0f52480571e2fcef6b70c5bf2038"
server_key_iv: "55460f0c37586ce208fc2d10ca809406"
server_key_version: "encryptionkey@0"
I20260810 19:32:50.918682 6793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:50.941740 6793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:50.946447 6793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:50.946790 6793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:50.949579 6793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:50.949690 6793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:50.949856 6793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:50.949949 6793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:32:50 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390370, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:50.975597 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.950576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:59397 (local address 127.29.233.190:39089)
0810 19:32:50.951077 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:50.951084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:50.952504 (+ 1420us) server_negotiation.cc:407] Connection header received
0810 19:32:50.955046 (+ 2542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:50.955053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:50.955153 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:50.955246 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:50.957945 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.959145 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.961680 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.962368 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.962508 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.967887 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:50.967896 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:50.967902 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:50.967946 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:50.971695 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.972755 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.972761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.972764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.972915 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.973467 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.973475 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.973480 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.973772 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:50.973944 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:50.974949 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0810 19:32:50.975177 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":251,"threads_started":1}
I20260810 19:32:50.976938 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.951277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39089 (local address 127.29.233.130:59397)
0810 19:32:50.952321 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:50.952382 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:50.954682 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:50.955842 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:50.955857 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:50.956775 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:50.957738 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.957761 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.959298 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.959304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.961476 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.961492 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.962627 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.962638 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.962955 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.963844 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:50.963892 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:50.967573 (+ 3681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:50.971862 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.971872 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.971897 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.972618 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.973047 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.973054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.973059 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.973339 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.973925 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:50.973933 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:50.974449 (+ 516us) client_negotiation.cc:769] Sending connection context
0810 19:32:50.974856 (+ 407us) client_negotiation.cc:240] Negotiation successful
0810 19:32:50.975468 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":475,"threads_started":1}
I20260810 19:32:50.999154 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.982841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:33711 (local address 127.29.233.190:39089)
0810 19:32:50.983037 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:50.983043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:50.983722 (+ 679us) server_negotiation.cc:407] Connection header received
0810 19:32:50.983911 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:50.983919 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:50.984019 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:50.984125 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:50.985723 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.986698 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.989063 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.989712 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.989859 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.992301 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:50.992309 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:50.992313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:50.992356 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:50.995654 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.996757 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.996764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.996767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.996948 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:50.997445 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:50.997454 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:50.997458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:50.997809 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:50.997997 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:50.998431 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 19:32:50.998666 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:32:50.999294 6898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:50.982861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39089 (local address 127.29.233.130:33711)
0810 19:32:50.983624 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:50.983639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:50.983791 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:50.984205 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:50.984213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:50.984990 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:50.985554 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.985572 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.986851 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.986856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.988821 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:50.988847 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:50.990072 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:50.990076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:50.990199 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:50.990933 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:50.990951 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:50.992009 (+ 1058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:50.995819 (+ 3810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.995829 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.995836 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.996534 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.997130 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:50.997137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:50.997140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:50.997311 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:50.997943 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:50.997947 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:50.998145 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 19:32:50.998335 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 19:32:50.998705 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":459,"threads_started":1}
I20260810 19:32:51.003364 6938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:37921 every 8 connection(s)
I20260810 19:32:51.003364 6793 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:37921
I20260810 19:32:51.004665 6793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:51.006846 6939 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39089
I20260810 19:32:51.007084 6939 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:51.007718 6939 heartbeater.cc:511] Master 127.29.233.190:39089 requested a full tablet report, sending...
I20260810 19:32:51.009073 6565 ts_manager.cc:194] Registered new tserver with Master: 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921)
I20260810 19:32:51.009909 6565 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:59397
I20260810 19:32:51.010442 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 6793
I20260810 19:32:51.010648 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:51.011266 30630 external_mini_cluster.cc:1428] Setting key f4902578622f5bc8d6c5415aef950a12
WARNING: no policy specified for oryx/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab.
Entry for principal oryx/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab.
I20260810 19:32:51.074501 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39089
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab
--principal=oryx/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:51.419665 6943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:51.420032 6943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:51.420140 6943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:51.427912 6943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:51.428031 6943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:51.428117 6943 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 19:32:51.428200 6943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:51.438561 6943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab
--principal=oryx/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.233.190:39089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:51.439936 6943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:51.441401 6943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:51.454692 6948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:51.454731 6951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:51.454767 6949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:51.455911 6943 server_base.cc:1034] running on GCE node
I20260810 19:32:51.456648 6943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:51.457789 6943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:51.459057 6943 hybrid_clock.cc:648] HybridClock initialized: now 1786390371458973 us; error 53 us; skew 500 ppm
I20260810 19:32:51.459419 6943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:51 dist-test-slave-tm1g krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390371, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:51.463215 6943 init.cc:377] Logged in from keytab as oryx/127.29.233.131@KRBTEST.COM (short username oryx)
I20260810 19:32:51.464645 6943 webserver.cc:460] Webserver started at http://127.29.233.131:41175/ using document root <none> and password file <none>
I20260810 19:32:51.465348 6943 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:51.465437 6943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:51.465716 6943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:51.468015 6789 heartbeater.cc:503] Master 127.29.233.190:39089 was elected leader, sending a full tablet report...
I20260810 19:32:51.468889 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "146022b3697542a091f33cea8dc27129"
format_stamp: "Formatted at 2026-08-10 19:32:51 on dist-test-slave-tm1g"
server_key: "4c6e738f38741f4357b6e6131ad80d85"
server_key_iv: "0e268ffd4839b07c0a1ba2a3ea388f1a"
server_key_version: "encryptionkey@0"
I20260810 19:32:51.469907 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "146022b3697542a091f33cea8dc27129"
format_stamp: "Formatted at 2026-08-10 19:32:51 on dist-test-slave-tm1g"
server_key: "4c6e738f38741f4357b6e6131ad80d85"
server_key_iv: "0e268ffd4839b07c0a1ba2a3ea388f1a"
server_key_version: "encryptionkey@0"
I20260810 19:32:51.475641 6943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 19:32:51.479154 6959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:51.480569 6943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:32:51.480737 6943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "146022b3697542a091f33cea8dc27129"
format_stamp: "Formatted at 2026-08-10 19:32:51 on dist-test-slave-tm1g"
server_key: "4c6e738f38741f4357b6e6131ad80d85"
server_key_iv: "0e268ffd4839b07c0a1ba2a3ea388f1a"
server_key_version: "encryptionkey@0"
I20260810 19:32:51.480962 6943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:51.505504 6943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:51.509781 6943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:51.510075 6943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:51.513041 6943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:51.513120 6943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:51.513291 6943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:51.513350 6943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:32:51 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390371, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:51.541700 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.513686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:42555 (local address 127.29.233.190:39089)
0810 19:32:51.514292 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.514320 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.515558 (+ 1238us) server_negotiation.cc:407] Connection header received
0810 19:32:51.518794 (+ 3236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.518803 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.518951 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.519070 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:51.522695 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.523950 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.527252 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.527732 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.527893 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.533346 (+ 5453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:51.533352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:51.533355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:51.533394 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:51.537265 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:51.538455 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:51.538464 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:51.538469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:51.538710 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:51.539312 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:51.539318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:51.539321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:51.539673 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:51.539840 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.541020 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.541265 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":358,"threads_started":1}
I20260810 19:32:51.543013 6970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.514411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39089 (local address 127.29.233.131:42555)
0810 19:32:51.515349 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.515414 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.518332 (+ 2918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.519926 (+ 1594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.519944 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.521146 (+ 1202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:51.522261 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.522411 (+ 150us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.524166 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.524176 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.527086 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.527101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.528064 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.528076 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.528486 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.529490 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:51.529534 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:51.533067 (+ 3533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:51.537465 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:51.537472 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:51.537486 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:51.538203 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:51.538849 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:51.538856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:51.538861 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:51.539173 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:51.539941 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:51.539951 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:51.540583 (+ 632us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.540901 (+ 318us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.541532 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":373,"threads_started":1}
I20260810 19:32:51.565258 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.549066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:40907 (local address 127.29.233.190:39089)
0810 19:32:51.549274 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.549282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.549939 (+ 657us) server_negotiation.cc:407] Connection header received
0810 19:32:51.550168 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.550172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.550244 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.550377 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:51.551916 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.552951 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.555621 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.556034 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.556132 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.558781 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:51.558786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:51.558789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:51.558822 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:51.562394 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:51.563258 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:51.563265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:51.563269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:51.563421 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:51.563789 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:51.563794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:51.563797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:51.564088 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:51.564270 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.564656 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.564882 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:32:51.565389 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.549086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39089 (local address 127.29.233.131:40907)
0810 19:32:51.549800 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.549835 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.550026 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.550464 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.550470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.551211 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:51.551742 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.551762 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.553111 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.553119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.555450 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.555471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.556246 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.556255 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.556495 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.557245 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:51.557273 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:51.558587 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:51.562524 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:51.562531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:51.562536 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:51.563121 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:51.563537 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:51.563543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:51.563547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:51.563677 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:51.564211 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:51.564215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:51.564417 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.564561 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.564883 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":349,"threads_started":1}
I20260810 19:32:51.570379 6943 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:44245
I20260810 19:32:51.570432 7088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:44245 every 8 connection(s)
I20260810 19:32:51.571687 6943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:51.573248 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 6943
I20260810 19:32:51.573450 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:51.574064 30630 external_mini_cluster.cc:1428] Setting key 664459a5125e35697d9ccc3930f227af
I20260810 19:32:51.573848 7089 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39089
I20260810 19:32:51.574793 7089 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:51.575628 7089 heartbeater.cc:511] Master 127.29.233.190:39089 requested a full tablet report, sending...
I20260810 19:32:51.577155 6564 ts_manager.cc:194] Registered new tserver with Master: 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245)
I20260810 19:32:51.578034 6564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:42555
I20260810 19:32:51.589593 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 19:32:51 dist-test-slave-tm1g krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390371, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 19:32:51 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:32:51 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 19:32:51.653503 6969 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:51.641845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.29.233.190:39089)
0810 19:32:51.642121 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.642131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.642429 (+ 298us) server_negotiation.cc:407] Connection header received
0810 19:32:51.642677 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.642684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.642766 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.642890 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:51.644379 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.645305 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.646774 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.647215 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.647362 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.653135 (+ 5773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48552: BlockingRecv error: recv got EOF from 127.0.0.1:48552 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 10 19:32:51 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390371, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:51.670907 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.656770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.29.233.190:39089)
0810 19:32:51.657033 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.657039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.657226 (+ 187us) server_negotiation.cc:407] Connection header received
0810 19:32:51.657365 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.657370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.657439 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.657522 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:51.658799 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.659518 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.660798 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.661244 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.661359 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.665258 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:51.665263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:51.665266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:51.665319 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:51.668293 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:51.668961 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:51.668965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:51.668968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:51.669096 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:51.669510 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:51.669514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:51.669516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:51.669790 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:51.669998 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.670271 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.670552 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260810 19:32:51.675127 6564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:51.682242 6564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:51.706102 7111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.695299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:37921 (local address 127.0.0.1:51036)
0810 19:32:51.696400 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.696441 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.696634 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.697284 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.697293 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.697331 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:51.697758 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.697774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.699814 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.699819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.703138 (+ 3319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.703153 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.705005 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.705012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.705206 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.705260 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.705426 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.705613 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":706,"threads_started":1}
I20260810 19:32:51.706629 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.695474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.29.233.130:37921)
0810 19:32:51.696674 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.696681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.696706 (+ 25us) server_negotiation.cc:407] Connection header received
0810 19:32:51.696827 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.696835 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.697074 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.697279 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:51.697977 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.699661 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.703316 (+ 3655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.704779 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.704963 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.705136 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.705484 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.705671 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":937,"threads_started":1}
I20260810 19:32:51.707048 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.697144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52716 (local address 127.29.233.129:39421)
0810 19:32:51.697900 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.697910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.698572 (+ 662us) server_negotiation.cc:407] Connection header received
0810 19:32:51.698680 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.698687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.698902 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.699088 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:51.699895 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.701413 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.704130 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.705306 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.705508 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.705654 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.706567 (+ 913us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.706748 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":419,"threads_started":1}
I20260810 19:32:51.707794 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.696822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39421 (local address 127.0.0.1:52716)
0810 19:32:51.698279 (+ 1457us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.698317 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.698445 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.699309 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.699317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.699351 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:51.699689 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.699696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.701561 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.701569 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.703982 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.703993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.705888 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.705896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.706102 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.706145 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.707294 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.707447 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"thread_start_us":1228,"threads_started":1}
I20260810 19:32:51.710489 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.699147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44245 (local address 127.0.0.1:51716)
0810 19:32:51.702018 (+ 2871us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.702036 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.702154 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.703317 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.703321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.703346 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:51.703732 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.703743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.705667 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.705672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.708051 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.708067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.709608 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.709615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.709783 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.709821 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.709940 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.710069 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2742,"mutex_wait_us":2,"thread_start_us":2585,"threads_started":1}
I20260810 19:32:51.710501 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.698748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.29.233.131:44245)
0810 19:32:51.702826 (+ 4078us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.702836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.702870 (+ 34us) server_negotiation.cc:407] Connection header received
0810 19:32:51.702965 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.702970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.703126 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.703309 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:51.703955 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.705499 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.708227 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.709399 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.709623 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.709780 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.709983 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.710139 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3907,"thread_start_us":3697,"threads_started":1}
I20260810 19:32:51.710614 6864 tablet_service.cc:1467] Processing CreateTablet for tablet 40a45f155ae74fcda0877f26452b4c8c (DEFAULT_TABLE table=test-table [id=93f73eacce77427cb4a7870859b8f02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:51.711277 6711 tablet_service.cc:1467] Processing CreateTablet for tablet 40a45f155ae74fcda0877f26452b4c8c (DEFAULT_TABLE table=test-table [id=93f73eacce77427cb4a7870859b8f02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:51.712738 6711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a45f155ae74fcda0877f26452b4c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:51.712922 6864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a45f155ae74fcda0877f26452b4c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:51.714102 7017 tablet_service.cc:1467] Processing CreateTablet for tablet 40a45f155ae74fcda0877f26452b4c8c (DEFAULT_TABLE table=test-table [id=93f73eacce77427cb4a7870859b8f02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:51.715444 7017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a45f155ae74fcda0877f26452b4c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:51.727739 7118 tablet_bootstrap.cc:492] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Bootstrap starting.
I20260810 19:32:51.729673 7119 tablet_bootstrap.cc:492] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Bootstrap starting.
I20260810 19:32:51.731887 7118 tablet_bootstrap.cc:654] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:51.733460 7118 log.cc:824] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:51.733517 7119 tablet_bootstrap.cc:654] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:51.734864 7119 log.cc:824] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:51.735054 7121 tablet_bootstrap.cc:492] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Bootstrap starting.
I20260810 19:32:51.737134 7118 tablet_bootstrap.cc:492] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: No bootstrap required, opened a new log
I20260810 19:32:51.737500 7118 ts_tablet_manager.cc:1397] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 19:32:51.738634 7121 tablet_bootstrap.cc:654] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:51.738664 7119 tablet_bootstrap.cc:492] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: No bootstrap required, opened a new log
I20260810 19:32:51.738956 7119 ts_tablet_manager.cc:1397] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 19:32:51.739861 7121 log.cc:824] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:51.751202 7119 raft_consensus.cc:348] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.752149 7119 raft_consensus.cc:374] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:51.752373 7119 raft_consensus.cc:729] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc11565f399d42b6b28391317d2aac65, State: Initialized, Role: FOLLOWER
I20260810 19:32:51.754439 7118 raft_consensus.cc:348] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.755117 7118 raft_consensus.cc:374] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:51.755232 7118 raft_consensus.cc:729] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af8509fd6b04d6299428eaaf0353dcf, State: Initialized, Role: FOLLOWER
I20260810 19:32:51.755955 7118 consensus_queue.cc:260] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.756760 7121 tablet_bootstrap.cc:492] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: No bootstrap required, opened a new log
I20260810 19:32:51.757119 7121 ts_tablet_manager.cc:1397] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.010s
I20260810 19:32:51.757201 6939 heartbeater.cc:503] Master 127.29.233.190:39089 was elected leader, sending a full tablet report...
I20260810 19:32:51.758489 7118 ts_tablet_manager.cc:1428] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Time spent starting tablet: real 0.021s user 0.016s sys 0.000s
I20260810 19:32:51.758258 7119 consensus_queue.cc:260] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.765175 7119 ts_tablet_manager.cc:1428] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Time spent starting tablet: real 0.026s user 0.019s sys 0.000s
I20260810 19:32:51.766563 7121 raft_consensus.cc:348] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.767089 7121 raft_consensus.cc:374] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:51.767215 7121 raft_consensus.cc:729] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146022b3697542a091f33cea8dc27129, State: Initialized, Role: FOLLOWER
I20260810 19:32:51.767895 7121 consensus_queue.cc:260] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.769429 7089 heartbeater.cc:503] Master 127.29.233.190:39089 was elected leader, sending a full tablet report...
I20260810 19:32:51.770025 7121 ts_tablet_manager.cc:1428] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260810 19:32:51.823545 7090 tablet.cc:2369] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:32:51.901654 7125 raft_consensus.cc:482] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:51.901988 7125 raft_consensus.cc:504] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.903741 7125 leader_election.cc:288] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245)
I20260810 19:32:51.908905 7127 raft_consensus.cc:482] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:51.909230 7127 raft_consensus.cc:504] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.911067 7127 leader_election.cc:288] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921), fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421)
I20260810 19:32:51.912846 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.904225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:39961 (local address 127.29.233.129:39421)
0810 19:32:51.904535 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.904542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.904997 (+ 455us) server_negotiation.cc:407] Connection header received
0810 19:32:51.905316 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.905321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.905411 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.905486 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:51.906280 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.907584 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.910451 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.911440 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.911600 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.911675 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.912341 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.912476 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260810 19:32:51.913949 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.905242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:59727 (local address 127.29.233.131:44245)
0810 19:32:51.905474 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.905478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.905651 (+ 173us) server_negotiation.cc:407] Connection header received
0810 19:32:51.905809 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.905812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.905886 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.905965 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:51.906672 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.907826 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.910405 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.911752 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.911891 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.911966 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.913593 (+ 1627us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.913722 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 19:32:51.914600 7128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.904112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39421 (local address 127.29.233.130:39961)
0810 19:32:51.904875 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.904898 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.905088 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.905698 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.905702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.905727 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:51.906120 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.906131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.907747 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.907751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.910210 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.910221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.911859 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.911865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.912082 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.912116 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.913776 (+ 1660us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.914011 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":60,"thread_start_us":358,"threads_started":1}
I20260810 19:32:51.915578 7129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.904844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44245 (local address 127.29.233.130:59727)
0810 19:32:51.905557 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.905573 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.905692 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.906015 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.906020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.906049 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:51.906509 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.906519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.907988 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.907995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.910206 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.910221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.913222 (+ 3001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.913230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.913405 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.913439 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.914435 (+ 996us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.914584 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":1,"thread_start_us":313,"threads_started":1}
I20260810 19:32:51.915797 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65" is_pre_election: true
I20260810 19:32:51.917017 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146022b3697542a091f33cea8dc27129" is_pre_election: true
I20260810 19:32:51.917507 7038 raft_consensus.cc:2454] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8af8509fd6b04d6299428eaaf0353dcf in term 0.
I20260810 19:32:51.918449 6812 leader_election.cc:302] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, 8af8509fd6b04d6299428eaaf0353dcf; no voters:
I20260810 19:32:51.918941 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.911894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:35333 (local address 127.29.233.130:37921)
0810 19:32:51.912137 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.912144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.912162 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:32:51.912221 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.912226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.912294 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.912381 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:51.912918 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.914065 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.916632 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.917464 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.917625 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.917691 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.918426 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.918599 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:32:51.919185 6970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.911255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:37921 (local address 127.29.233.131:35333)
0810 19:32:51.911481 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.911501 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.911618 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.912532 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.912538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.912566 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:51.912764 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.912771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.914236 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.914241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.916241 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.916256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.917621 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.917628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.917849 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.917880 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.917964 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.918086 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 19:32:51.916445 6738 raft_consensus.cc:2454] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8af8509fd6b04d6299428eaaf0353dcf in term 0.
I20260810 19:32:51.919078 7125 raft_consensus.cc:2781] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:51.919807 7125 raft_consensus.cc:482] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:51.919957 7125 raft_consensus.cc:3037] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:51.921473 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf" is_pre_election: true
I20260810 19:32:51.922916 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.912078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:56313 (local address 127.29.233.129:39421)
0810 19:32:51.913227 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:51.913235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:51.913255 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:51.913323 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:51.913330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:51.913501 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:51.913610 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:51.915922 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.918228 (+ 2306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.920532 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.921874 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.922055 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.922120 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:51.922469 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 19:32:51.922625 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":8455,"threads_started":1}
I20260810 19:32:51.922957 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:51.911628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39421 (local address 127.29.233.131:56313)
0810 19:32:51.912386 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:51.912403 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:51.912516 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:51.915293 (+ 2777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:51.915301 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:51.915332 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:51.915693 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.915705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.918424 (+ 2719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.918428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.920354 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:51.920366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:51.922089 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:51.922096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:51.922280 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:51.922351 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 19:32:51.922449 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 19:32:51.922602 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":688,"threads_started":1}
I20260810 19:32:51.923830 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65" is_pre_election: true
I20260810 19:32:51.924077 6738 raft_consensus.cc:2454] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 0.
I20260810 19:32:51.924036 7125 raft_consensus.cc:504] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.924453 6886 raft_consensus.cc:2379] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 146022b3697542a091f33cea8dc27129 in current term 1: Already voted for candidate 8af8509fd6b04d6299428eaaf0353dcf in this term.
I20260810 19:32:51.924551 6962 leader_election.cc:302] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, fc11565f399d42b6b28391317d2aac65; no voters:
I20260810 19:32:51.924703 7125 leader_election.cc:288] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 election: Requested vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245)
I20260810 19:32:51.925028 7127 raft_consensus.cc:2781] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:51.925173 7127 raft_consensus.cc:482] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:51.925282 7127 raft_consensus.cc:3037] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:51.925482 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65"
I20260810 19:32:51.925550 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146022b3697542a091f33cea8dc27129"
I20260810 19:32:51.925678 6738 raft_consensus.cc:3037] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:51.928630 7127 raft_consensus.cc:504] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.928879 6738 raft_consensus.cc:2454] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8af8509fd6b04d6299428eaaf0353dcf in term 1.
I20260810 19:32:51.929170 7038 raft_consensus.cc:2379] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8af8509fd6b04d6299428eaaf0353dcf in current term 1: Already voted for candidate 146022b3697542a091f33cea8dc27129 in this term.
I20260810 19:32:51.929278 6812 leader_election.cc:302] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8af8509fd6b04d6299428eaaf0353dcf, fc11565f399d42b6b28391317d2aac65; no voters:
I20260810 19:32:51.929390 7127 leader_election.cc:288] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 election: Requested vote from peers 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921), fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421)
I20260810 19:32:51.929752 7125 raft_consensus.cc:2781] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:51.929816 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65"
I20260810 19:32:51.929862 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a45f155ae74fcda0877f26452b4c8c" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf"
I20260810 19:32:51.930089 6738 raft_consensus.cc:2379] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 146022b3697542a091f33cea8dc27129 in current term 1: Already voted for candidate 8af8509fd6b04d6299428eaaf0353dcf in this term.
I20260810 19:32:51.930490 7125 raft_consensus.cc:686] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 1 LEADER]: Becoming Leader. State: Replica: 8af8509fd6b04d6299428eaaf0353dcf, State: Running, Role: LEADER
I20260810 19:32:51.931082 7125 consensus_queue.cc:237] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:51.933754 6962 leader_election.cc:302] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 146022b3697542a091f33cea8dc27129; no voters: 8af8509fd6b04d6299428eaaf0353dcf, fc11565f399d42b6b28391317d2aac65
I20260810 19:32:51.934166 7127 raft_consensus.cc:2726] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 19:32:51.935737 6565 catalog_manager.cc:5495] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130). New cstate: current_term: 1 leader_uuid: "8af8509fd6b04d6299428eaaf0353dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } health_report { overall_health: UNKNOWN } } }
W20260810 19:32:51.961282 6790 tablet.cc:2369] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:32:52.007146 6940 tablet.cc:2369] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:32:52.008090 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.001919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.29.233.130:37921)
0810 19:32:52.002124 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.002130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.002150 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:52.002236 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.002241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.002370 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.002513 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:52.002891 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.004069 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.005454 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.006120 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.006238 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.006727 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.007416 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.007547 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.007697 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.007786 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 19:32:52.016850 7038 raft_consensus.cc:1261] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Refusing update from remote peer 8af8509fd6b04d6299428eaaf0353dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.016965 6738 raft_consensus.cc:1261] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Refusing update from remote peer 8af8509fd6b04d6299428eaaf0353dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.017737 7132 consensus_queue.cc:1035] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.018172 7125 consensus_queue.cc:1035] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.034132 7139 mvcc.cc:205] Tried to move back new op lower bound from 7317054963770908672 to 7317054963433865216. Current Snapshot: MvccSnapshot[applied={T|T < 7317054963770908672}]
I20260810 19:32:52.047211 6564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:52.047551 6564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:52.051591 6564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Sending DeleteTablet for 3 replicas of tablet 40a45f155ae74fcda0877f26452b4c8c
I20260810 19:32:52.052996 6864 tablet_service.cc:1514] Processing DeleteTablet for tablet 40a45f155ae74fcda0877f26452b4c8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51036
I20260810 19:32:52.052992 6711 tablet_service.cc:1514] Processing DeleteTablet for tablet 40a45f155ae74fcda0877f26452b4c8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52716
I20260810 19:32:52.053787 7017 tablet_service.cc:1514] Processing DeleteTablet for tablet 40a45f155ae74fcda0877f26452b4c8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51716
I20260810 19:32:52.054587 6564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:52.055495 6564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:52.057379 7150 tablet_replica.cc:330] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: stopping tablet replica
I20260810 19:32:52.057890 7150 raft_consensus.cc:2229] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:52.058856 7151 tablet_replica.cc:330] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: stopping tablet replica
I20260810 19:32:52.058977 7149 tablet_replica.cc:330] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: stopping tablet replica
I20260810 19:32:52.059293 7151 raft_consensus.cc:2229] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:52.059372 7149 raft_consensus.cc:2229] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:52.059832 7151 raft_consensus.cc:2258] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.059859 7149 raft_consensus.cc:2258] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.060314 7150 raft_consensus.cc:2258] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.063743 7149 ts_tablet_manager.cc:1905] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.064482 7017 tablet_service.cc:1467] Processing CreateTablet for tablet d1c3cbb38bae4224a074ec995272f55d (DEFAULT_TABLE table=test-table [id=72f36dd144db4c528179a82a7657aaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.064958 7017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c3cbb38bae4224a074ec995272f55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.065029 6864 tablet_service.cc:1467] Processing CreateTablet for tablet d1c3cbb38bae4224a074ec995272f55d (DEFAULT_TABLE table=test-table [id=72f36dd144db4c528179a82a7657aaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.066108 6864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c3cbb38bae4224a074ec995272f55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.067811 7151 ts_tablet_manager.cc:1905] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.071328 7149 ts_tablet_manager.cc:1918] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:52.071555 7149 log.cc:1196] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/40a45f155ae74fcda0877f26452b4c8c
I20260810 19:32:52.071822 6711 tablet_service.cc:1467] Processing CreateTablet for tablet d1c3cbb38bae4224a074ec995272f55d (DEFAULT_TABLE table=test-table [id=72f36dd144db4c528179a82a7657aaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.072175 7149 ts_tablet_manager.cc:1939] T 40a45f155ae74fcda0877f26452b4c8c P fc11565f399d42b6b28391317d2aac65: Deleting consensus metadata
I20260810 19:32:52.072278 6711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c3cbb38bae4224a074ec995272f55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.072647 7121 tablet_bootstrap.cc:492] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: Bootstrap starting.
I20260810 19:32:52.074077 7121 tablet_bootstrap.cc:654] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.074134 6550 catalog_manager.cc:4841] TS fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421): tablet 40a45f155ae74fcda0877f26452b4c8c (table test-table [id=93f73eacce77427cb4a7870859b8f02f]) successfully deleted
I20260810 19:32:52.076109 7121 tablet_bootstrap.cc:492] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: No bootstrap required, opened a new log
I20260810 19:32:52.076267 7121 ts_tablet_manager.cc:1397] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 19:32:52.076727 7118 tablet_bootstrap.cc:492] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: Bootstrap starting.
I20260810 19:32:52.077059 7121 raft_consensus.cc:348] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.077271 7121 raft_consensus.cc:374] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.077360 7121 raft_consensus.cc:729] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146022b3697542a091f33cea8dc27129, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.077402 7150 ts_tablet_manager.cc:1905] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.077679 7121 consensus_queue.cc:260] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.078679 7118 tablet_bootstrap.cc:654] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.079370 7121 ts_tablet_manager.cc:1428] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 19:32:52.084182 7119 tablet_bootstrap.cc:492] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: Bootstrap starting.
I20260810 19:32:52.084964 7150 ts_tablet_manager.cc:1918] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:52.085166 7150 log.cc:1196] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/40a45f155ae74fcda0877f26452b4c8c
I20260810 19:32:52.085757 7150 ts_tablet_manager.cc:1939] T 40a45f155ae74fcda0877f26452b4c8c P 8af8509fd6b04d6299428eaaf0353dcf: Deleting consensus metadata
I20260810 19:32:52.086037 7119 tablet_bootstrap.cc:654] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.087513 7151 ts_tablet_manager.cc:1918] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:52.087703 7151 log.cc:1196] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/40a45f155ae74fcda0877f26452b4c8c
I20260810 19:32:52.087736 6550 catalog_manager.cc:4841] TS 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921): tablet 40a45f155ae74fcda0877f26452b4c8c (table test-table [id=93f73eacce77427cb4a7870859b8f02f]) successfully deleted
I20260810 19:32:52.087993 7118 tablet_bootstrap.cc:492] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: No bootstrap required, opened a new log
I20260810 19:32:52.088178 7118 ts_tablet_manager.cc:1397] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260810 19:32:52.088243 7151 ts_tablet_manager.cc:1939] T 40a45f155ae74fcda0877f26452b4c8c P 146022b3697542a091f33cea8dc27129: Deleting consensus metadata
I20260810 19:32:52.089054 7118 raft_consensus.cc:348] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.089286 7118 raft_consensus.cc:374] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.089476 7118 raft_consensus.cc:729] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af8509fd6b04d6299428eaaf0353dcf, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.089784 7118 consensus_queue.cc:260] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.090078 6550 catalog_manager.cc:4841] TS 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245): tablet 40a45f155ae74fcda0877f26452b4c8c (table test-table [id=93f73eacce77427cb4a7870859b8f02f]) successfully deleted
I20260810 19:32:52.090521 7118 ts_tablet_manager.cc:1428] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 19:32:52.099944 7119 tablet_bootstrap.cc:492] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: No bootstrap required, opened a new log
I20260810 19:32:52.100136 7119 ts_tablet_manager.cc:1397] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.012s
I20260810 19:32:52.100791 7119 raft_consensus.cc:348] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.101001 7119 raft_consensus.cc:374] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.101099 7119 raft_consensus.cc:729] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc11565f399d42b6b28391317d2aac65, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.101373 7119 consensus_queue.cc:260] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.101878 7119 ts_tablet_manager.cc:1428] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 19:32:52.104108 7142 raft_consensus.cc:482] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:52.104314 7142 raft_consensus.cc:504] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.105078 7142 leader_election.cc:288] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245)
I20260810 19:32:52.105619 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c3cbb38bae4224a074ec995272f55d" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146022b3697542a091f33cea8dc27129" is_pre_election: true
I20260810 19:32:52.105623 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c3cbb38bae4224a074ec995272f55d" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65" is_pre_election: true
I20260810 19:32:52.105852 6738 raft_consensus.cc:2454] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8af8509fd6b04d6299428eaaf0353dcf in term 0.
I20260810 19:32:52.105885 7038 raft_consensus.cc:2454] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8af8509fd6b04d6299428eaaf0353dcf in term 0.
I20260810 19:32:52.106366 6812 leader_election.cc:302] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, 8af8509fd6b04d6299428eaaf0353dcf; no voters:
I20260810 19:32:52.106832 7142 raft_consensus.cc:2781] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:52.106967 7142 raft_consensus.cc:482] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:52.107071 7142 raft_consensus.cc:3037] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.110445 7142 raft_consensus.cc:504] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.111299 7142 leader_election.cc:288] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 election: Requested vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245)
I20260810 19:32:52.111512 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c3cbb38bae4224a074ec995272f55d" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65"
I20260810 19:32:52.111714 6738 raft_consensus.cc:3037] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.111866 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c3cbb38bae4224a074ec995272f55d" candidate_uuid: "8af8509fd6b04d6299428eaaf0353dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146022b3697542a091f33cea8dc27129"
I20260810 19:32:52.112131 7038 raft_consensus.cc:3037] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.113873 6738 raft_consensus.cc:2454] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8af8509fd6b04d6299428eaaf0353dcf in term 1.
I20260810 19:32:52.114341 6812 leader_election.cc:302] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8af8509fd6b04d6299428eaaf0353dcf, fc11565f399d42b6b28391317d2aac65; no voters:
I20260810 19:32:52.114670 7142 raft_consensus.cc:2781] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:52.114857 7142 raft_consensus.cc:686] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 LEADER]: Becoming Leader. State: Replica: 8af8509fd6b04d6299428eaaf0353dcf, State: Running, Role: LEADER
I20260810 19:32:52.115247 7038 raft_consensus.cc:2454] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8af8509fd6b04d6299428eaaf0353dcf in term 1.
I20260810 19:32:52.115162 7142 consensus_queue.cc:237] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.117941 6564 catalog_manager.cc:5495] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130). New cstate: current_term: 1 leader_uuid: "8af8509fd6b04d6299428eaaf0353dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } health_report { overall_health: UNKNOWN } } }
Aug 10 19:32:52 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390369, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.190@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:52.176541 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.157045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.29.233.190:39089)
0810 19:32:52.157291 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.157299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.157929 (+ 630us) server_negotiation.cc:407] Connection header received
0810 19:32:52.158124 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.158128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.158189 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.158272 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:52.159850 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.160608 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.162586 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.163005 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.163105 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.170502 (+ 7397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:52.170507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:52.170510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:52.170542 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:52.173493 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:52.174439 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:52.174443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:52.174445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:52.174570 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:52.174973 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:52.174976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:52.174978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:52.175241 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:52.175361 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.175878 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.176215 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":82}
I20260810 19:32:52.176594 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.157045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39089 (local address 127.0.0.1:48578)
0810 19:32:52.157806 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:52.157828 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:52.158006 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:52.158366 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:52.158372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:52.159320 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:52.159675 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.159688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.160754 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.160760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.162441 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.162457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.163194 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.163201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.163383 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.167250 (+ 3867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:52.167276 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:52.170232 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:52.173626 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:52.173640 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:52.173649 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:52.174256 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:52.174684 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:52.174689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:52.174693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:52.174860 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:52.175369 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:52.175378 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:52.175592 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 19:32:52.175765 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 19:32:52.176088 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":387,"threads_started":1}
I20260810 19:32:52.182516 6564 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:48578:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 19:32:52.183854 6564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:52.191821 6711 tablet_service.cc:1467] Processing CreateTablet for tablet aa1aabf531c34d618b79de94eccb972d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9351da9e00d4cfc8f933b51f6add858]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:32:52.192255 7017 tablet_service.cc:1467] Processing CreateTablet for tablet aa1aabf531c34d618b79de94eccb972d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9351da9e00d4cfc8f933b51f6add858]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:32:52.192507 6711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa1aabf531c34d618b79de94eccb972d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.192762 7017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa1aabf531c34d618b79de94eccb972d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.195001 6864 tablet_service.cc:1467] Processing CreateTablet for tablet aa1aabf531c34d618b79de94eccb972d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9351da9e00d4cfc8f933b51f6add858]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 19:32:52.195626 6864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa1aabf531c34d618b79de94eccb972d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.200031 7121 tablet_bootstrap.cc:492] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: Bootstrap starting.
I20260810 19:32:52.201762 7121 tablet_bootstrap.cc:654] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.202193 7118 tablet_bootstrap.cc:492] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: Bootstrap starting.
I20260810 19:32:52.203507 7119 tablet_bootstrap.cc:492] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: Bootstrap starting.
I20260810 19:32:52.203972 7118 tablet_bootstrap.cc:654] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.205374 7119 tablet_bootstrap.cc:654] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.212765 7118 tablet_bootstrap.cc:492] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: No bootstrap required, opened a new log
I20260810 19:32:52.212999 7118 ts_tablet_manager.cc:1397] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260810 19:32:52.214076 7118 raft_consensus.cc:348] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.214494 7118 raft_consensus.cc:374] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.214587 7118 raft_consensus.cc:729] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af8509fd6b04d6299428eaaf0353dcf, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.214890 7118 consensus_queue.cc:260] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.215392 7142 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.215696 7142 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:52.216151 7119 tablet_bootstrap.cc:492] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: No bootstrap required, opened a new log
I20260810 19:32:52.216373 7119 ts_tablet_manager.cc:1397] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260810 19:32:52.216974 7121 tablet_bootstrap.cc:492] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: No bootstrap required, opened a new log
I20260810 19:32:52.217090 7118 ts_tablet_manager.cc:1428] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 19:32:52.217161 7121 ts_tablet_manager.cc:1397] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.008s
I20260810 19:32:52.217366 7119 raft_consensus.cc:348] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.217674 7119 raft_consensus.cc:374] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.217800 7119 raft_consensus.cc:729] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc11565f399d42b6b28391317d2aac65, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.217929 7121 raft_consensus.cc:348] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.218176 7121 raft_consensus.cc:374] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.218277 7121 raft_consensus.cc:729] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146022b3697542a091f33cea8dc27129, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.218214 7119 consensus_queue.cc:260] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.218562 7121 consensus_queue.cc:260] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.219075 7119 ts_tablet_manager.cc:1428] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 19:32:52.218979 7127 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.219004 7126 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.219295 7127 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:52.219393 7126 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:52.220275 7121 ts_tablet_manager.cc:1428] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 19:32:52.319181 7127 raft_consensus.cc:482] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:52.319446 7127 raft_consensus.cc:504] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.320057 7127 leader_election.cc:288] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921)
I20260810 19:32:52.320768 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1aabf531c34d618b79de94eccb972d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf" is_pre_election: true
I20260810 19:32:52.320808 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1aabf531c34d618b79de94eccb972d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65" is_pre_election: true
I20260810 19:32:52.321128 6738 raft_consensus.cc:2454] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 0.
I20260810 19:32:52.321097 6886 raft_consensus.cc:2454] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 0.
I20260810 19:32:52.321681 6962 leader_election.cc:302] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, 8af8509fd6b04d6299428eaaf0353dcf; no voters:
I20260810 19:32:52.322041 7127 raft_consensus.cc:2781] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:52.322153 7127 raft_consensus.cc:482] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:52.322268 7127 raft_consensus.cc:3037] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.324586 7127 raft_consensus.cc:504] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.325217 7127 leader_election.cc:288] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 election: Requested vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921)
I20260810 19:32:52.325745 6738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1aabf531c34d618b79de94eccb972d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65"
I20260810 19:32:52.325899 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa1aabf531c34d618b79de94eccb972d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf"
I20260810 19:32:52.326012 6738 raft_consensus.cc:3037] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.326154 6886 raft_consensus.cc:3037] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.328388 6738 raft_consensus.cc:2454] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 1.
I20260810 19:32:52.328912 6962 leader_election.cc:302] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, fc11565f399d42b6b28391317d2aac65; no voters:
I20260810 19:32:52.329252 7127 raft_consensus.cc:2781] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:52.329661 6886 raft_consensus.cc:2454] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 1.
I20260810 19:32:52.329814 7127 raft_consensus.cc:686] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [term 1 LEADER]: Becoming Leader. State: Replica: 146022b3697542a091f33cea8dc27129, State: Running, Role: LEADER
I20260810 19:32:52.330197 7127 consensus_queue.cc:237] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.332422 7166 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: TxnStatusTablet state changed. Reason: New leader 146022b3697542a091f33cea8dc27129. Latest consensus state: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.332753 7166 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:32:52.333600 7168 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 19:32:52.333855 7168 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 19:32:52.334656 6564 catalog_manager.cc:5495] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 reported cstate change: term changed from 0 to 1, leader changed from <none> to 146022b3697542a091f33cea8dc27129 (127.29.233.131). New cstate: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } health_report { overall_health: HEALTHY } } }
I20260810 19:32:52.372572 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.366666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.29.233.131:44245)
0810 19:32:52.366852 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.366859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.366878 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:32:52.367178 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.367184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.367303 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.367471 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:52.367846 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.368695 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.370033 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.370554 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.370732 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.371286 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.371817 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.372004 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.372155 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.372244 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:32:52.372732 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.366545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44245 (local address 127.0.0.1:51730)
0810 19:32:52.366788 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:52.366808 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:52.366964 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:52.367489 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:52.367493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:52.367505 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:52.367708 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.367717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.368830 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.368836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.369889 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.369901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.370749 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.370755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.370967 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.371050 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.372000 (+ 950us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.372025 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 19:32:52.372146 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 19:32:52.372393 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 19:32:52.377190 6886 raft_consensus.cc:1261] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Refusing update from remote peer 146022b3697542a091f33cea8dc27129: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.377307 6738 raft_consensus.cc:1261] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Refusing update from remote peer 146022b3697542a091f33cea8dc27129: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.378075 7127 consensus_queue.cc:1035] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.378885 7166 consensus_queue.cc:1035] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.390702 7142 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: TxnStatusTablet state changed. Reason: New leader 146022b3697542a091f33cea8dc27129. Latest consensus state: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.391151 7142 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:52.396265 7137 mvcc.cc:205] Tried to move back new op lower bound from 7317054965248495616 to 7317054965068898304. Current Snapshot: MvccSnapshot[applied={T|T < 7317054965248495616}]
I20260810 19:32:52.399695 7127 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.400029 7127 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:32:52.400686 7142 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.400972 7142 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P 8af8509fd6b04d6299428eaaf0353dcf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:52.406492 7126 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: TxnStatusTablet state changed. Reason: New leader 146022b3697542a091f33cea8dc27129. Latest consensus state: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.406823 7126 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:52.411365 7127 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.411775 7127 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P 146022b3697542a091f33cea8dc27129: This TxnStatusTablet replica's current role is: LEADER
I20260810 19:32:52.414423 7126 tablet_replica.cc:439] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } } }
I20260810 19:32:52.414700 7126 tablet_replica.cc:442] T aa1aabf531c34d618b79de94eccb972d P fc11565f399d42b6b28391317d2aac65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 19:32:52.440243 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.432269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:44771 (local address 127.29.233.131:44245)
0810 19:32:52.432530 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.432536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.433207 (+ 671us) server_negotiation.cc:407] Connection header received
0810 19:32:52.433415 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.433423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.433538 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.433641 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:52.434175 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.435365 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.437786 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.438544 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.438695 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.439302 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.439463 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.439565 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.439742 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.439847 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 19:32:52.440425 7175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.432113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44245 (local address 127.29.233.130:44771)
0810 19:32:52.433076 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:52.433097 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:52.433258 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:52.433683 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:52.433688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:52.433701 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:52.433992 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.434015 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.435675 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.435681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.437593 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.437615 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.438785 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.438796 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.438989 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.439073 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.439580 (+ 507us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.439596 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 19:32:52.439694 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 19:32:52.439928 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":619,"threads_started":1}
I20260810 19:32:52.453289 7038 raft_consensus.cc:1261] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Refusing update from remote peer 8af8509fd6b04d6299428eaaf0353dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.453493 6737 raft_consensus.cc:1261] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Refusing update from remote peer 8af8509fd6b04d6299428eaaf0353dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.454030 7125 consensus_queue.cc:1035] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.454703 7125 consensus_queue.cc:1035] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.459528 7136 mvcc.cc:205] Tried to move back new op lower bound from 7317054965561552896 to 7317054964187226112. Current Snapshot: MvccSnapshot[applied={T|T < 7317054965561552896}]
I20260810 19:32:52.512529 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.503748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:48245 (local address 127.29.233.130:37921)
0810 19:32:52.503961 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.503967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.504845 (+ 878us) server_negotiation.cc:407] Connection header received
0810 19:32:52.505063 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.505070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.505154 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.505269 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:52.505858 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.507044 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.509929 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.510718 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.510829 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.511615 (+ 786us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.511760 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.511830 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.512068 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.512164 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 19:32:52.513628 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.503972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:37921 (local address 127.29.233.131:48245)
0810 19:32:52.504703 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:52.504724 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:52.504891 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:52.505310 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:52.505315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:52.505329 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:52.505661 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.505673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.507226 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.507231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.509727 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.509744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.511016 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.511022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.511218 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.511299 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.511919 (+ 620us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.511929 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 19:32:52.513041 (+ 1112us) client_negotiation.cc:240] Negotiation successful
0810 19:32:52.513217 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":386,"threads_started":1}
I20260810 19:32:52.569293 6564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:52.569572 6564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:52.573462 6564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Sending DeleteTablet for 3 replicas of tablet d1c3cbb38bae4224a074ec995272f55d
I20260810 19:32:52.574760 6711 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c3cbb38bae4224a074ec995272f55d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52716
I20260810 19:32:52.574926 6864 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c3cbb38bae4224a074ec995272f55d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51036
I20260810 19:32:52.575225 7150 tablet_replica.cc:330] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: stopping tablet replica
I20260810 19:32:52.575394 7017 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c3cbb38bae4224a074ec995272f55d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51716
I20260810 19:32:52.575457 7150 raft_consensus.cc:2229] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:52.575717 7151 tablet_replica.cc:330] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: stopping tablet replica
I20260810 19:32:52.575798 7150 raft_consensus.cc:2258] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.575968 7151 raft_consensus.cc:2229] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:52.576180 7151 raft_consensus.cc:2258] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.577236 7151 ts_tablet_manager.cc:1905] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.577245 7150 ts_tablet_manager.cc:1905] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.584602 7151 ts_tablet_manager.cc:1918] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:52.584774 7151 log.cc:1196] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/d1c3cbb38bae4224a074ec995272f55d
I20260810 19:32:52.585352 7151 ts_tablet_manager.cc:1939] T d1c3cbb38bae4224a074ec995272f55d P 146022b3697542a091f33cea8dc27129: Deleting consensus metadata
I20260810 19:32:52.585745 7150 ts_tablet_manager.cc:1918] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:52.585948 7150 log.cc:1196] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/d1c3cbb38bae4224a074ec995272f55d
I20260810 19:32:52.586463 7150 ts_tablet_manager.cc:1939] T d1c3cbb38bae4224a074ec995272f55d P 8af8509fd6b04d6299428eaaf0353dcf: Deleting consensus metadata
I20260810 19:32:52.586637 6550 catalog_manager.cc:4841] TS 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245): tablet d1c3cbb38bae4224a074ec995272f55d (table test-table [id=72f36dd144db4c528179a82a7657aaa4]) successfully deleted
I20260810 19:32:52.587541 6550 catalog_manager.cc:4841] TS 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921): tablet d1c3cbb38bae4224a074ec995272f55d (table test-table [id=72f36dd144db4c528179a82a7657aaa4]) successfully deleted
I20260810 19:32:52.598712 7182 tablet_replica.cc:330] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: stopping tablet replica
I20260810 19:32:52.599054 7182 raft_consensus.cc:2229] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:52.599370 7182 raft_consensus.cc:2258] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.600630 7182 ts_tablet_manager.cc:1905] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.608310 7182 ts_tablet_manager.cc:1918] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:52.618458 7182 log.cc:1196] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/d1c3cbb38bae4224a074ec995272f55d
I20260810 19:32:52.619088 7182 ts_tablet_manager.cc:1939] T d1c3cbb38bae4224a074ec995272f55d P fc11565f399d42b6b28391317d2aac65: Deleting consensus metadata
I20260810 19:32:52.620460 6550 catalog_manager.cc:4841] TS fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421): tablet d1c3cbb38bae4224a074ec995272f55d (table test-table [id=72f36dd144db4c528179a82a7657aaa4]) successfully deleted
Aug 10 19:32:52 dist-test-slave-tm1g krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 19:32:52 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:32:52 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 19:32:52.671916 6969 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:32:52.659947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.29.233.190:39089)
0810 19:32:52.660140 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.660146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.660724 (+ 578us) server_negotiation.cc:407] Connection header received
0810 19:32:52.660907 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.660913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.661011 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.661126 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:52.662909 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.664136 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.665684 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.666370 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.666511 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.671625 (+ 5114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48594: BlockingRecv error: recv got EOF from 127.0.0.1:48594 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 19:32:52 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM
I20260810 19:32:52.692278 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.675042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.29.233.190:39089)
0810 19:32:52.675285 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.675291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.675674 (+ 383us) server_negotiation.cc:407] Connection header received
0810 19:32:52.675821 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.675831 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.675909 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.676007 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:52.677640 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.678514 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.680353 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.680781 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.680894 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.685590 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:52.685598 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:52.685604 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:52.685650 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:52.689217 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:52.689932 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:52.689938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:52.689942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:52.690093 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:52.690639 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:52.690647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:52.690652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:52.691046 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:52.691322 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.691640 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.691877 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:32:52.696422 6564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:32:52.697232 6564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:52.704624 6711 tablet_service.cc:1467] Processing CreateTablet for tablet 773f4ac5192a449f8b32b95efb1579dc (DEFAULT_TABLE table=test-table [id=adf020aacc584a1e8e4bee81a750930c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.705152 6711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 773f4ac5192a449f8b32b95efb1579dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.707306 6864 tablet_service.cc:1467] Processing CreateTablet for tablet 773f4ac5192a449f8b32b95efb1579dc (DEFAULT_TABLE table=test-table [id=adf020aacc584a1e8e4bee81a750930c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.707779 6864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 773f4ac5192a449f8b32b95efb1579dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.710855 7017 tablet_service.cc:1467] Processing CreateTablet for tablet 773f4ac5192a449f8b32b95efb1579dc (DEFAULT_TABLE table=test-table [id=adf020aacc584a1e8e4bee81a750930c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.711268 7017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 773f4ac5192a449f8b32b95efb1579dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.714589 7118 tablet_bootstrap.cc:492] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: Bootstrap starting.
I20260810 19:32:52.716280 7118 tablet_bootstrap.cc:654] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.717644 7119 tablet_bootstrap.cc:492] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: Bootstrap starting.
I20260810 19:32:52.718660 7118 tablet_bootstrap.cc:492] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: No bootstrap required, opened a new log
I20260810 19:32:52.718798 7118 ts_tablet_manager.cc:1397] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 19:32:52.719374 7119 tablet_bootstrap.cc:654] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.719583 7118 raft_consensus.cc:348] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.719878 7118 raft_consensus.cc:374] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.719982 7118 raft_consensus.cc:729] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af8509fd6b04d6299428eaaf0353dcf, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.720211 7118 consensus_queue.cc:260] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.720811 7118 ts_tablet_manager.cc:1428] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 19:32:52.727634 7199 tablet_bootstrap.cc:492] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: Bootstrap starting.
I20260810 19:32:52.727852 7119 tablet_bootstrap.cc:492] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: No bootstrap required, opened a new log
I20260810 19:32:52.728081 7119 ts_tablet_manager.cc:1397] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260810 19:32:52.728937 7119 raft_consensus.cc:348] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.729225 7119 raft_consensus.cc:374] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.729279 7199 tablet_bootstrap.cc:654] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.729393 7119 raft_consensus.cc:729] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc11565f399d42b6b28391317d2aac65, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.729664 7119 consensus_queue.cc:260] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.732797 7119 ts_tablet_manager.cc:1428] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 19:32:52.736024 7199 tablet_bootstrap.cc:492] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: No bootstrap required, opened a new log
I20260810 19:32:52.736233 7199 ts_tablet_manager.cc:1397] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260810 19:32:52.736995 7199 raft_consensus.cc:348] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.737238 7199 raft_consensus.cc:374] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.737351 7199 raft_consensus.cc:729] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146022b3697542a091f33cea8dc27129, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.737579 7199 consensus_queue.cc:260] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.738178 7199 ts_tablet_manager.cc:1428] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 19:32:52.833341 7126 raft_consensus.cc:482] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:52.833636 7126 raft_consensus.cc:504] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.835520 7126 leader_election.cc:288] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921), 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245)
I20260810 19:32:52.843169 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.836059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:47527 (local address 127.29.233.130:37921)
0810 19:32:52.836264 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.836270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.836547 (+ 277us) server_negotiation.cc:407] Connection header received
0810 19:32:52.836821 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.836828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.836903 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.836983 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:52.837731 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.838862 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.841017 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.841899 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.842059 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.842151 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.842698 (+ 547us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.842850 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 19:32:52.844069 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.835831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:37921 (local address 127.29.233.129:47527)
0810 19:32:52.836428 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:52.836448 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:52.836664 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:52.837203 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:52.837208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:52.837239 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:52.837561 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.837571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.839002 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.839008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.840866 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.840878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.842276 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.842282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.842498 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.842533 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 19:32:52.843474 (+ 941us) client_negotiation.cc:240] Negotiation successful
0810 19:32:52.843615 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":47,"thread_start_us":262,"threads_started":1}
I20260810 19:32:52.844883 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.836567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44245 (local address 127.29.233.129:38583)
0810 19:32:52.837137 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:52.837152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:52.837314 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:52.837657 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:52.837661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:52.837684 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:32:52.838021 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.838029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.839389 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.839394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.841672 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:52.841688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.843991 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.843999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:52.844216 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.844249 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 19:32:52.844355 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 19:32:52.844552 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":365,"threads_started":1}
I20260810 19:32:52.844892 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773f4ac5192a449f8b32b95efb1579dc" candidate_uuid: "fc11565f399d42b6b28391317d2aac65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf" is_pre_election: true
I20260810 19:32:52.845247 6886 raft_consensus.cc:2454] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc11565f399d42b6b28391317d2aac65 in term 0.
I20260810 19:32:52.845763 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.836710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:38583 (local address 127.29.233.131:44245)
0810 19:32:52.836913 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.836921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.837246 (+ 325us) server_negotiation.cc:407] Connection header received
0810 19:32:52.837471 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.837476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.837549 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.837643 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:32:52.838183 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.839236 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.841869 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.843571 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.843730 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.843806 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.844399 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.844528 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 19:32:52.845829 6661 leader_election.cc:302] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8af8509fd6b04d6299428eaaf0353dcf, fc11565f399d42b6b28391317d2aac65; no voters:
I20260810 19:32:52.846284 7126 raft_consensus.cc:2781] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:52.846432 7126 raft_consensus.cc:482] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:52.846352 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773f4ac5192a449f8b32b95efb1579dc" candidate_uuid: "fc11565f399d42b6b28391317d2aac65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146022b3697542a091f33cea8dc27129" is_pre_election: true
I20260810 19:32:52.846536 7126 raft_consensus.cc:3037] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.846642 7038 raft_consensus.cc:2454] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc11565f399d42b6b28391317d2aac65 in term 0.
I20260810 19:32:52.849920 7126 raft_consensus.cc:504] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.850808 7126 leader_election.cc:288] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [CANDIDATE]: Term 1 election: Requested vote from peers 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921), 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245)
I20260810 19:32:52.851315 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773f4ac5192a449f8b32b95efb1579dc" candidate_uuid: "fc11565f399d42b6b28391317d2aac65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf"
I20260810 19:32:52.851475 7038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "773f4ac5192a449f8b32b95efb1579dc" candidate_uuid: "fc11565f399d42b6b28391317d2aac65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146022b3697542a091f33cea8dc27129"
I20260810 19:32:52.851644 6886 raft_consensus.cc:3037] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.851752 7038 raft_consensus.cc:3037] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:52.855044 6886 raft_consensus.cc:2454] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc11565f399d42b6b28391317d2aac65 in term 1.
I20260810 19:32:52.855091 7038 raft_consensus.cc:2454] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc11565f399d42b6b28391317d2aac65 in term 1.
I20260810 19:32:52.855566 6661 leader_election.cc:302] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, fc11565f399d42b6b28391317d2aac65; no voters:
I20260810 19:32:52.855932 7126 raft_consensus.cc:2781] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:52.856388 7126 raft_consensus.cc:686] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 1 LEADER]: Becoming Leader. State: Replica: fc11565f399d42b6b28391317d2aac65, State: Running, Role: LEADER
I20260810 19:32:52.857136 7126 consensus_queue.cc:237] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.861261 6564 catalog_manager.cc:5495] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc11565f399d42b6b28391317d2aac65 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "fc11565f399d42b6b28391317d2aac65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } health_report { overall_health: UNKNOWN } } }
I20260810 19:32:52.885347 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:52.879022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.29.233.129:39421)
0810 19:32:52.879625 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:52.879629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:52.879649 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:32:52.879734 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:52.879737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:52.879811 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:52.879991 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:52.880464 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.881262 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.882859 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:52.883311 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:52.883453 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:52.883935 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:52.884586 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:52.884750 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:52.884808 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 19:32:52.884891 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":294,"threads_started":1}
I20260810 19:32:52.889315 7038 raft_consensus.cc:1261] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Refusing update from remote peer fc11565f399d42b6b28391317d2aac65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.889444 6886 raft_consensus.cc:1261] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Refusing update from remote peer fc11565f399d42b6b28391317d2aac65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:52.890074 7203 consensus_queue.cc:1035] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.890538 7126 consensus_queue.cc:1035] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:52.906543 6563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:52.906831 6563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:52.909722 6563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Sending DeleteTablet for 3 replicas of tablet 773f4ac5192a449f8b32b95efb1579dc
I20260810 19:32:52.910746 6711 tablet_service.cc:1514] Processing DeleteTablet for tablet 773f4ac5192a449f8b32b95efb1579dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52716
I20260810 19:32:52.910751 6864 tablet_service.cc:1514] Processing DeleteTablet for tablet 773f4ac5192a449f8b32b95efb1579dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51036
I20260810 19:32:52.911054 7182 tablet_replica.cc:330] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: stopping tablet replica
I20260810 19:32:52.911075 7150 tablet_replica.cc:330] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: stopping tablet replica
I20260810 19:32:52.911254 7150 raft_consensus.cc:2229] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:52.911283 7182 raft_consensus.cc:2229] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:52.911521 7150 raft_consensus.cc:2258] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.911679 6563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 19:32:52.911943 7017 tablet_service.cc:1514] Processing DeleteTablet for tablet 773f4ac5192a449f8b32b95efb1579dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:52 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51716
I20260810 19:32:52.912236 7151 tablet_replica.cc:330] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: stopping tablet replica
W20260810 19:32:52.912336 6563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:32:52.912454 7151 raft_consensus.cc:2229] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:52.912706 7151 raft_consensus.cc:2258] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.912940 7150 ts_tablet_manager.cc:1905] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.914057 7151 ts_tablet_manager.cc:1905] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.918895 7182 raft_consensus.cc:2258] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:52.919231 7151 ts_tablet_manager.cc:1918] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:52.919427 7151 log.cc:1196] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/773f4ac5192a449f8b32b95efb1579dc
I20260810 19:32:52.919687 7150 ts_tablet_manager.cc:1918] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:52.919824 7150 log.cc:1196] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/773f4ac5192a449f8b32b95efb1579dc
I20260810 19:32:52.920174 7151 ts_tablet_manager.cc:1939] T 773f4ac5192a449f8b32b95efb1579dc P 146022b3697542a091f33cea8dc27129: Deleting consensus metadata
I20260810 19:32:52.920279 7150 ts_tablet_manager.cc:1939] T 773f4ac5192a449f8b32b95efb1579dc P 8af8509fd6b04d6299428eaaf0353dcf: Deleting consensus metadata
I20260810 19:32:52.920492 7182 ts_tablet_manager.cc:1905] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:52.921463 6550 catalog_manager.cc:4841] TS 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921): tablet 773f4ac5192a449f8b32b95efb1579dc (table test-table [id=adf020aacc584a1e8e4bee81a750930c]) successfully deleted
I20260810 19:32:52.921919 6550 catalog_manager.cc:4841] TS 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245): tablet 773f4ac5192a449f8b32b95efb1579dc (table test-table [id=adf020aacc584a1e8e4bee81a750930c]) successfully deleted
I20260810 19:32:52.925963 7182 ts_tablet_manager.cc:1918] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:32:52.926177 7182 log.cc:1196] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/773f4ac5192a449f8b32b95efb1579dc
I20260810 19:32:52.926836 7182 ts_tablet_manager.cc:1939] T 773f4ac5192a449f8b32b95efb1579dc P fc11565f399d42b6b28391317d2aac65: Deleting consensus metadata
I20260810 19:32:52.929167 6550 catalog_manager.cc:4841] TS fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421): tablet 773f4ac5192a449f8b32b95efb1579dc (table test-table [id=adf020aacc584a1e8e4bee81a750930c]) successfully deleted
I20260810 19:32:52.931005 6711 tablet_service.cc:1467] Processing CreateTablet for tablet 5be7eb0c973f497ea24a9fa4570db18d (DEFAULT_TABLE table=test-table [id=4cb0b848d0094678b95ca06272cd15a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.931452 6711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be7eb0c973f497ea24a9fa4570db18d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.931450 7017 tablet_service.cc:1467] Processing CreateTablet for tablet 5be7eb0c973f497ea24a9fa4570db18d (DEFAULT_TABLE table=test-table [id=4cb0b848d0094678b95ca06272cd15a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.931476 6864 tablet_service.cc:1467] Processing CreateTablet for tablet 5be7eb0c973f497ea24a9fa4570db18d (DEFAULT_TABLE table=test-table [id=4cb0b848d0094678b95ca06272cd15a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:32:52.931905 7017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be7eb0c973f497ea24a9fa4570db18d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.931939 6864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be7eb0c973f497ea24a9fa4570db18d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:52.938555 7199 tablet_bootstrap.cc:492] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: Bootstrap starting.
I20260810 19:32:52.939218 7118 tablet_bootstrap.cc:492] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: Bootstrap starting.
I20260810 19:32:52.939687 7119 tablet_bootstrap.cc:492] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: Bootstrap starting.
I20260810 19:32:52.940246 7199 tablet_bootstrap.cc:654] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.940918 7118 tablet_bootstrap.cc:654] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.941443 7119 tablet_bootstrap.cc:654] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:52.943804 7119 tablet_bootstrap.cc:492] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: No bootstrap required, opened a new log
I20260810 19:32:52.943993 7119 ts_tablet_manager.cc:1397] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 19:32:52.944180 7118 tablet_bootstrap.cc:492] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: No bootstrap required, opened a new log
I20260810 19:32:52.944334 7118 ts_tablet_manager.cc:1397] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 19:32:52.944808 7119 raft_consensus.cc:348] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.945080 7119 raft_consensus.cc:374] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.945185 7119 raft_consensus.cc:729] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc11565f399d42b6b28391317d2aac65, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.945124 7118 raft_consensus.cc:348] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.945400 7118 raft_consensus.cc:374] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.945487 7118 raft_consensus.cc:729] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af8509fd6b04d6299428eaaf0353dcf, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.945458 7119 consensus_queue.cc:260] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.945761 7118 consensus_queue.cc:260] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.946105 7119 ts_tablet_manager.cc:1428] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 19:32:52.948428 7118 ts_tablet_manager.cc:1428] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260810 19:32:52.952584 7199 tablet_bootstrap.cc:492] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: No bootstrap required, opened a new log
I20260810 19:32:52.952785 7199 ts_tablet_manager.cc:1397] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260810 19:32:52.953404 7199 raft_consensus.cc:348] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.953640 7199 raft_consensus.cc:374] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:52.953734 7199 raft_consensus.cc:729] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146022b3697542a091f33cea8dc27129, State: Initialized, Role: FOLLOWER
I20260810 19:32:52.953992 7199 consensus_queue.cc:260] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:52.954546 7199 ts_tablet_manager.cc:1428] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 19:32:53.192070 7127 raft_consensus.cc:482] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:32:53.192325 7127 raft_consensus.cc:504] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:53.193089 7127 leader_election.cc:288] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921)
I20260810 19:32:53.193843 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be7eb0c973f497ea24a9fa4570db18d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf" is_pre_election: true
I20260810 19:32:53.193857 6737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be7eb0c973f497ea24a9fa4570db18d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65" is_pre_election: true
I20260810 19:32:53.194159 6886 raft_consensus.cc:2454] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 0.
I20260810 19:32:53.194260 6737 raft_consensus.cc:2454] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 0.
I20260810 19:32:53.194729 6962 leader_election.cc:302] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, 8af8509fd6b04d6299428eaaf0353dcf; no voters:
I20260810 19:32:53.195035 7127 raft_consensus.cc:2781] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:32:53.195142 7127 raft_consensus.cc:482] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:32:53.195252 7127 raft_consensus.cc:3037] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:53.197285 7127 raft_consensus.cc:504] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:53.197909 7127 leader_election.cc:288] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 election: Requested vote from peers fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421), 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921)
I20260810 19:32:53.198506 6737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be7eb0c973f497ea24a9fa4570db18d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc11565f399d42b6b28391317d2aac65"
I20260810 19:32:53.198577 6886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be7eb0c973f497ea24a9fa4570db18d" candidate_uuid: "146022b3697542a091f33cea8dc27129" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8509fd6b04d6299428eaaf0353dcf"
I20260810 19:32:53.198756 6737 raft_consensus.cc:3037] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:53.198856 6886 raft_consensus.cc:3037] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:53.201288 6737 raft_consensus.cc:2454] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 1.
I20260810 19:32:53.201783 6962 leader_election.cc:302] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146022b3697542a091f33cea8dc27129, fc11565f399d42b6b28391317d2aac65; no voters:
I20260810 19:32:53.201952 6886 raft_consensus.cc:2454] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146022b3697542a091f33cea8dc27129 in term 1.
I20260810 19:32:53.202055 7127 raft_consensus.cc:2781] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:53.202579 7127 raft_consensus.cc:686] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 1 LEADER]: Becoming Leader. State: Replica: 146022b3697542a091f33cea8dc27129, State: Running, Role: LEADER
I20260810 19:32:53.202906 7127 consensus_queue.cc:237] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } }
I20260810 19:32:53.205826 6562 catalog_manager.cc:5495] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 reported cstate change: term changed from 0 to 1, leader changed from <none> to 146022b3697542a091f33cea8dc27129 (127.29.233.131). New cstate: current_term: 1 leader_uuid: "146022b3697542a091f33cea8dc27129" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146022b3697542a091f33cea8dc27129" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44245 } health_report { overall_health: HEALTHY } } }
I20260810 19:32:53.256667 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:53.250784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.29.233.131:44245)
0810 19:32:53.250976 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:53.250983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:53.251005 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:32:53.251069 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:53.251074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:53.251163 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:53.251270 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:53.251691 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.252825 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.254917 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.255380 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.255484 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:53.256026 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:53.256140 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:53.256209 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:53.256410 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 19:32:53.256476 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:32:53.270869 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:53.265641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:53933 (local address 127.29.233.131:44245)
0810 19:32:53.265837 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:53.265845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:53.266201 (+ 356us) server_negotiation.cc:407] Connection header received
0810 19:32:53.266447 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:53.266453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:53.266514 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:53.266602 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:32:53.267020 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.267778 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.269058 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.269491 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.269633 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:53.270026 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:32:53.270132 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:32:53.270229 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:53.270394 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 19:32:53.270459 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":50}
I20260810 19:32:53.270900 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:53.265540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44245 (local address 127.29.233.131:53933)
0810 19:32:53.266101 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:53.266116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:53.266276 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:53.266624 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:53.266628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:53.266639 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 19:32:53.266890 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:53.266898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.267927 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.267932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:53.268923 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:53.268934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.269646 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.269651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:53.269830 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:53.269869 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 19:32:53.270227 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 19:32:53.270239 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 19:32:53.270373 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 19:32:53.270499 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":1,"thread_start_us":338,"threads_started":1}
I20260810 19:32:53.280921 6737 raft_consensus.cc:1261] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Refusing update from remote peer 146022b3697542a091f33cea8dc27129: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:53.280956 6886 raft_consensus.cc:1261] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Refusing update from remote peer 146022b3697542a091f33cea8dc27129: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:32:53.281664 7216 consensus_queue.cc:1035] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8af8509fd6b04d6299428eaaf0353dcf" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 37921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:53.282018 7214 consensus_queue.cc:1035] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc11565f399d42b6b28391317d2aac65" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:32:53.343225 6562 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:53.343557 6562 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:32:53.347373 6562 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e19bcc5592f4dd6b42b511bf036fdef: Sending DeleteTablet for 3 replicas of tablet 5be7eb0c973f497ea24a9fa4570db18d
I20260810 19:32:53.348569 6711 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be7eb0c973f497ea24a9fa4570db18d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:53 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52716
I20260810 19:32:53.348588 7017 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be7eb0c973f497ea24a9fa4570db18d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:53 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51716
I20260810 19:32:53.348886 7151 tablet_replica.cc:330] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: stopping tablet replica
I20260810 19:32:53.348910 7182 tablet_replica.cc:330] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: stopping tablet replica
I20260810 19:32:53.349086 7182 raft_consensus.cc:2229] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:53.349107 7151 raft_consensus.cc:2229] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:32:53.349283 7182 raft_consensus.cc:2258] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:53.349634 7151 raft_consensus.cc:2258] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:53.351045 7182 ts_tablet_manager.cc:1905] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:53.354954 7151 ts_tablet_manager.cc:1905] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:53.356007 6864 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be7eb0c973f497ea24a9fa4570db18d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:32:53 UTC) from {username='oryx', principal='oryx/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:51036
I20260810 19:32:53.357206 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 6642
I20260810 19:32:53.358166 7150 tablet_replica.cc:330] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: stopping tablet replica
I20260810 19:32:53.358381 7150 raft_consensus.cc:2229] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:32:53.358575 7150 raft_consensus.cc:2258] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:32:53.360116 7150 ts_tablet_manager.cc:1905] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:32:53.363325 7151 ts_tablet_manager.cc:1918] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:53.363485 7151 log.cc:1196] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/5be7eb0c973f497ea24a9fa4570db18d
I20260810 19:32:53.363891 7151 ts_tablet_manager.cc:1939] T 5be7eb0c973f497ea24a9fa4570db18d P 146022b3697542a091f33cea8dc27129: Deleting consensus metadata
I20260810 19:32:53.364984 6550 catalog_manager.cc:4841] TS 146022b3697542a091f33cea8dc27129 (127.29.233.131:44245): tablet 5be7eb0c973f497ea24a9fa4570db18d (table test-table [id=4cb0b848d0094678b95ca06272cd15a3]) successfully deleted
I20260810 19:32:53.365526 7182 ts_tablet_manager.cc:1918] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:53.365681 7182 log.cc:1196] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/5be7eb0c973f497ea24a9fa4570db18d
I20260810 19:32:53.366118 7182 ts_tablet_manager.cc:1939] T 5be7eb0c973f497ea24a9fa4570db18d P fc11565f399d42b6b28391317d2aac65: Deleting consensus metadata
I20260810 19:32:53.367311 6550 catalog_manager.cc:4841] TS fc11565f399d42b6b28391317d2aac65 (127.29.233.129:39421): tablet 5be7eb0c973f497ea24a9fa4570db18d (table test-table [id=4cb0b848d0094678b95ca06272cd15a3]) successfully deleted
I20260810 19:32:53.368693 7150 ts_tablet_manager.cc:1918] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 19:32:53.369014 7150 log.cc:1196] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/5be7eb0c973f497ea24a9fa4570db18d
I20260810 19:32:53.369493 7150 ts_tablet_manager.cc:1939] T 5be7eb0c973f497ea24a9fa4570db18d P 8af8509fd6b04d6299428eaaf0353dcf: Deleting consensus metadata
I20260810 19:32:53.370731 6550 catalog_manager.cc:4841] TS 8af8509fd6b04d6299428eaaf0353dcf (127.29.233.130:37921): tablet 5be7eb0c973f497ea24a9fa4570db18d (table test-table [id=4cb0b848d0094678b95ca06272cd15a3]) successfully deleted
Aug 10 19:32:53 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.129@KRBTEST.COM
I20260810 19:32:53.383355 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:53.358844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.29.233.129:39421)
0810 19:32:53.359071 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:53.359079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:53.365751 (+ 6672us) server_negotiation.cc:407] Connection header received
0810 19:32:53.366024 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:53.366029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:53.366100 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:53.366168 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:53.367953 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.369026 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.370922 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.371590 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.371685 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:53.376942 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:53.376954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:53.376970 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:53.377007 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:53.380515 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:53.381250 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:53.381258 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:53.381260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:53.381402 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:53.381806 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:53.381811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:53.381813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:53.382160 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:53.382531 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:53.382801 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 19:32:53.383035 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:32:53.384029 6784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:53.513001 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 6642
I20260810 19:32:53.549088 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 6793
Aug 10 19:32:53 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.130@KRBTEST.COM
I20260810 19:32:53.565587 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:53.549956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51050 (local address 127.29.233.130:37921)
0810 19:32:53.550519 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:53.550523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:53.550539 (+ 16us) server_negotiation.cc:407] Connection header received
0810 19:32:53.550595 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:53.550598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:53.550665 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:53.550788 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:53.552111 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.552863 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.554210 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.554842 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.555009 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:53.559555 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:53.559572 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:53.559589 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:53.559626 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:53.562675 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:53.563369 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:53.563377 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:53.563380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:53.563512 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:53.563901 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:53.563907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:53.563910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:53.564294 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:53.564625 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:53.564935 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 19:32:53.565197 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":281,"threads_started":1}
I20260810 19:32:53.566347 6934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:53.714924 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 6793
I20260810 19:32:53.741894 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 6943
Aug 10 19:32:53 dist-test-slave-tm1g krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.233.131@KRBTEST.COM
I20260810 19:32:53.763509 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:53.743101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.29.233.131:44245)
0810 19:32:53.743397 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:53.743404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:53.743426 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:32:53.743614 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:53.743642 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:53.743781 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:53.744061 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:53.746132 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.747493 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.749325 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:53.749802 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:53.749911 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:53.755680 (+ 5769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:53.755703 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:53.755726 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:53.755791 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:53.760129 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:53.760963 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:53.760972 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:53.760976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:53.761152 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:53.761609 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:53.761614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:53.761617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:53.762079 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:53.762457 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:53.762888 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 19:32:53.763139 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 19:32:53.764298 7084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:53.921432 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 6943
I20260810 19:32:53.951506 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 6531
I20260810 19:32:53.952868 6607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:54.071388 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 6531
2026-08-10T19:32:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6206 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:54 dist-test-slave-tm1g krb5kdc[7236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:54 dist-test-slave-tm1g krb5kdc[7236](info): set up 2 sockets
Aug 10 19:32:54 dist-test-slave-tm1g krb5kdc[7236](info): commencing operation
krb5kdc: starting...
W20260810 19:32:55.243297 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:55 dist-test-slave-tm1g krb5kdc[7236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:55.516215 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37797
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:37797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:55.864126 7250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:55.864463 7250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:55.864640 7250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:55.871989 7250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:55.872138 7250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:55.872236 7250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:55.872311 7250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:55.872363 7250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:55.882176 7250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:37797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:55.883546 7250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:55.884814 7250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:55.897925 7256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:55.897897 7259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:55.897966 7257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:55.899420 7250 server_base.cc:1034] running on GCE node
I20260810 19:32:55.900223 7250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:55.901584 7250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:55.902786 7250 hybrid_clock.cc:648] HybridClock initialized: now 1786390375902740 us; error 31 us; skew 500 ppm
I20260810 19:32:55.903136 7250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T19:32:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1894 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:32:56 dist-test-slave-tm1g krb5kdc[7267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:32:56 dist-test-slave-tm1g krb5kdc[7267](info): set up 2 sockets
Aug 10 19:32:56 dist-test-slave-tm1g krb5kdc[7267](info): commencing operation
krb5kdc: starting...
W20260810 19:32:57.166715 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:32:57 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:32:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:32:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:32:57.441850 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37269
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:37269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:57.787717 7281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:57.788065 7281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:57.788228 7281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:57.795420 7281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:32:57.795567 7281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:57.795678 7281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:57.795781 7281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:32:57.795858 7281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:32:57.805612 7281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:37269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:57.807052 7281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:57.808435 7281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:57.820575 7288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:57.820549 7290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:57.820647 7287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:57.821710 7281 server_base.cc:1034] running on GCE node
I20260810 19:32:57.822547 7281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:57.823946 7281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:57.825165 7281 hybrid_clock.cc:648] HybridClock initialized: now 1786390377825104 us; error 57 us; skew 500 ppm
I20260810 19:32:57.825539 7281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:57 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390377, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:57.832408 7281 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:32:57.834019 7281 webserver.cc:460] Webserver started at http://127.29.233.190:45851/ using document root <none> and password file <none>
I20260810 19:32:57.834808 7281 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:57.834959 7281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:57.835294 7281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:57.838392 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade"
format_stamp: "Formatted at 2026-08-10 19:32:57 on dist-test-slave-tm1g"
server_key: "45b414897687df93d3e259be884bfd8e"
server_key_iv: "06ab30845d3b1087ef16511f92cab4b9"
server_key_version: "encryptionkey@0"
I20260810 19:32:57.839191 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade"
format_stamp: "Formatted at 2026-08-10 19:32:57 on dist-test-slave-tm1g"
server_key: "45b414897687df93d3e259be884bfd8e"
server_key_iv: "06ab30845d3b1087ef16511f92cab4b9"
server_key_version: "encryptionkey@0"
I20260810 19:32:57.844419 7281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:32:57.847994 7297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:57.849323 7281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:32:57.849510 7281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade"
format_stamp: "Formatted at 2026-08-10 19:32:57 on dist-test-slave-tm1g"
server_key: "45b414897687df93d3e259be884bfd8e"
server_key_iv: "06ab30845d3b1087ef16511f92cab4b9"
server_key_version: "encryptionkey@0"
I20260810 19:32:57.849711 7281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:57.863911 7281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:57.868432 7281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:57.868731 7281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:57.887801 7281 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:37269
I20260810 19:32:57.887820 7352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:37269 every 8 connection(s)
I20260810 19:32:57.889261 7281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:32:57.891125 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 7281
I20260810 19:32:57.891340 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:32:57.892033 30630 external_mini_cluster.cc:1428] Setting key 6f9e3ea35cadf5b9f9c87394a261d7a4
I20260810 19:32:57.893496 7353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:32:57.907123 7353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Bootstrap starting.
Aug 10 19:32:57 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:57.910909 7353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Neither blocks nor log segments found. Creating new log.
I20260810 19:32:57.912283 7353 log.cc:824] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Log is configured to *not* fsync() on all Append() calls
I20260810 19:32:57.916090 7353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: No bootstrap required, opened a new log
I20260810 19:32:57.918139 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:57.894100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.29.233.190:37269)
0810 19:32:57.894979 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:57.894994 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:57.895046 (+ 52us) server_negotiation.cc:407] Connection header received
0810 19:32:57.896954 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:57.896981 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:57.897479 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:57.897899 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:57.899404 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:57.901456 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:57.902973 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:57.904043 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:57.904240 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:57.908747 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:57.908760 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:57.908784 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:57.908845 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:57.913024 (+ 4179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:57.914079 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:57.914090 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:57.914104 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:57.914324 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:57.914856 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:57.914863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:57.914866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:57.915634 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:57.915913 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:57.916358 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 19:32:57.916828 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":386,"threads_started":1}
I20260810 19:32:57.925041 7353 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:32:57.925398 7353 raft_consensus.cc:374] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:32:57.925522 7353 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c7510cc4a4ce9a4eb1aecb5762ade, State: Initialized, Role: FOLLOWER
I20260810 19:32:57.926192 7353 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:32:57.926540 7353 raft_consensus.cc:388] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:32:57.926684 7353 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:32:57.926877 7353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:32:57.929207 7353 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:32:57.929708 7353 leader_election.cc:302] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f77c7510cc4a4ce9a4eb1aecb5762ade; no voters:
I20260810 19:32:57.930375 7353 leader_election.cc:288] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:32:57.930547 7358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:32:57.931397 7358 raft_consensus.cc:686] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 LEADER]: Becoming Leader. State: Replica: f77c7510cc4a4ce9a4eb1aecb5762ade, State: Running, Role: LEADER
I20260810 19:32:57.931911 7358 consensus_queue.cc:237] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:32:57.932533 7353 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:32:57.935145 7360 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader f77c7510cc4a4ce9a4eb1aecb5762ade. Latest consensus state: current_term: 1 leader_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } } }
I20260810 19:32:57.935673 7360 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:57.935135 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } } }
I20260810 19:32:57.936321 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: This master's current role is: LEADER
I20260810 19:32:57.938791 7366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:32:57.943622 7366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:32:57.952421 7366 catalog_manager.cc:1347] Generated new cluster ID: 19ef589069c74fa5829508337df58644
I20260810 19:32:57.952548 7366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:32:57.961257 7366 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:32:57.962594 7366 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:32:57.973732 7366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Generated new TSK 0
I20260810 19:32:57.974637 7366 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:32:58.042865 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37269
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:58.393399 7379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:58.393687 7379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:58.393862 7379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:58.401708 7379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:58.401813 7379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:58.402019 7379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:32:58.411959 7379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:58.413795 7379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:58.415426 7379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:58.430655 7388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:58.430737 7384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:58.430734 7385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:58.432251 7379 server_base.cc:1034] running on GCE node
I20260810 19:32:58.432963 7379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:58.434080 7379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:58.435295 7379 hybrid_clock.cc:648] HybridClock initialized: now 1786390378435223 us; error 61 us; skew 500 ppm
I20260810 19:32:58.435644 7379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:58 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390378, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:58.439055 7379 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:32:58.440649 7379 webserver.cc:460] Webserver started at http://127.29.233.129:39705/ using document root <none> and password file <none>
I20260810 19:32:58.441378 7379 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:58.441524 7379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:58.441846 7379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:58.444916 7379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "fc6c302390f94495bbe9b29dd2b807d5"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "add81d70ca6538bb85d66f55ebb9c37a"
server_key_iv: "532bd3bf6375c8a08f030e254978ad71"
server_key_version: "encryptionkey@0"
I20260810 19:32:58.445765 7379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "fc6c302390f94495bbe9b29dd2b807d5"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "add81d70ca6538bb85d66f55ebb9c37a"
server_key_iv: "532bd3bf6375c8a08f030e254978ad71"
server_key_version: "encryptionkey@0"
I20260810 19:32:58.451143 7379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 19:32:58.454769 7395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:58.456094 7379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:32:58.456271 7379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "fc6c302390f94495bbe9b29dd2b807d5"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "add81d70ca6538bb85d66f55ebb9c37a"
server_key_iv: "532bd3bf6375c8a08f030e254978ad71"
server_key_version: "encryptionkey@0"
I20260810 19:32:58.456471 7379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:58.474293 7379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:58.478899 7379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:58.479219 7379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:58.480203 7379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:58.481596 7379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:58.481702 7379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:58.481863 7379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:58.481940 7379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:58.516804 7379 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:34519
I20260810 19:32:58.516839 7517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:34519 every 8 connection(s)
I20260810 19:32:58.518122 7379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:32:58.522722 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 7379
I20260810 19:32:58.522933 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:32:58.523784 30630 external_mini_cluster.cc:1428] Setting key 87f2375ae04f1291affc457fc193e950
Aug 10 19:32:58 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390378, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:58.541544 7520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:58.520780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:36801 (local address 127.29.233.190:37269)
0810 19:32:58.521286 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:58.521293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:58.522185 (+ 892us) server_negotiation.cc:407] Connection header received
0810 19:32:58.524903 (+ 2718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:58.524907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:58.525013 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:58.525145 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:58.527080 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:58.527817 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:58.530442 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:58.530885 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:58.530996 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:58.535567 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:58.535572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:58.535576 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:58.535613 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:58.538211 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:58.539042 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:58.539047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:58.539050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:58.539180 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:58.539703 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:58.539707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:58.539709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:58.540055 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:58.540174 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:58.541085 (+ 911us) server_negotiation.cc:299] Negotiation successful
0810 19:32:58.541264 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":228,"threads_started":1}
I20260810 19:32:58.543161 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:58.521276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:36801)
0810 19:32:58.522010 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:58.522048 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:58.524553 (+ 2505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:58.525374 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:58.525381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:58.526062 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:58.526880 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:58.526910 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:58.528002 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:58.528010 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:58.530171 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:58.530189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:58.531127 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:58.531137 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:58.531481 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:58.532261 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:58.532300 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:58.535280 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:58.538421 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:58.538433 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:58.538451 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:58.538897 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:58.539305 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:58.539310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:58.539317 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:58.539574 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:58.540192 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:58.540204 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:58.540676 (+ 472us) client_negotiation.cc:769] Sending connection context
0810 19:32:58.540984 (+ 308us) client_negotiation.cc:240] Negotiation successful
0810 19:32:58.541589 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":269,"threads_started":1}
I20260810 19:32:58.544834 7518 heartbeater.cc:347] Connected to a master server at 127.29.233.190:37269
I20260810 19:32:58.545143 7518 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:58.545902 7518 heartbeater.cc:511] Master 127.29.233.190:37269 requested a full tablet report, sending...
I20260810 19:32:58.548106 7317 ts_manager.cc:194] Registered new tserver with Master: fc6c302390f94495bbe9b29dd2b807d5 (127.29.233.129:34519)
I20260810 19:32:58.549554 7317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:36801
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:32:58.590196 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37269
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:58.926402 7524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:58.926683 7524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:58.926779 7524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:58.934072 7524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:58.934181 7524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:58.934347 7524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:32:58.944321 7524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:58.945760 7524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:58.947121 7524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:58.960492 7531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:58.960412 7534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:58.960455 7532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:58.961796 7524 server_base.cc:1034] running on GCE node
I20260810 19:32:58.962474 7524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:58.963411 7524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:58.964654 7524 hybrid_clock.cc:648] HybridClock initialized: now 1786390378964610 us; error 55 us; skew 500 ppm
I20260810 19:32:58.964978 7524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:58 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390378, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:58.968114 7524 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:32:58.969640 7524 webserver.cc:460] Webserver started at http://127.29.233.130:36851/ using document root <none> and password file <none>
I20260810 19:32:58.970423 7524 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:58.970577 7524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:58.970902 7524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:58.973886 7524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "d57c378494074c5a8ca7c6d939f02b3e"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "398ba8438c769d4b88c17a7738c998f6"
server_key_iv: "674b8130243848033963fc22c28ce69b"
server_key_version: "encryptionkey@0"
I20260810 19:32:58.974766 7524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "d57c378494074c5a8ca7c6d939f02b3e"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "398ba8438c769d4b88c17a7738c998f6"
server_key_iv: "674b8130243848033963fc22c28ce69b"
server_key_version: "encryptionkey@0"
I20260810 19:32:58.979847 7524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:32:58.983301 7541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:58.984576 7524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:32:58.984745 7524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "d57c378494074c5a8ca7c6d939f02b3e"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "398ba8438c769d4b88c17a7738c998f6"
server_key_iv: "674b8130243848033963fc22c28ce69b"
server_key_version: "encryptionkey@0"
I20260810 19:32:58.984949 7524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:59.002040 7524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:59.006110 7524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:59.006436 7524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:59.007400 7524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:59.008754 7524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:59.008855 7524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:59.009006 7524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:59.009079 7524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:59.044003 7524 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:45997
I20260810 19:32:59.044025 7664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:45997 every 8 connection(s)
I20260810 19:32:59.045425 7524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:32:59.047422 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 7524
I20260810 19:32:59.047618 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:32:59.048265 30630 external_mini_cluster.cc:1428] Setting key 13a18269a65cb761a2eb505d12e3b2dc
Aug 10 19:32:59 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390378, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:59.067986 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:59.048272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:44631 (local address 127.29.233.190:37269)
0810 19:32:59.048773 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:59.048777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:59.050016 (+ 1239us) server_negotiation.cc:407] Connection header received
0810 19:32:59.052031 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:59.052036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:59.052127 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:59.052246 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:59.053959 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.055139 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.056713 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.057157 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.057298 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:59.061020 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:59.061030 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:59.061037 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:59.061085 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:59.064361 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.065357 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.065361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.065363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.065512 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.066052 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.066055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.066057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.066423 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:59.066553 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:59.067529 (+ 976us) server_negotiation.cc:299] Negotiation successful
0810 19:32:59.067724 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":290,"threads_started":1}
I20260810 19:32:59.069818 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:59.048933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:44631)
0810 19:32:59.049813 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:59.049891 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:59.051749 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:59.052504 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:59.052512 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:59.053107 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:59.053742 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:59.053760 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.055292 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.055296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:59.056564 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:59.056577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.057413 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.057424 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:59.057670 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:59.058435 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:59.058463 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:59.060800 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:59.064517 (+ 3717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:59.064529 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:59.064550 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:59.065209 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:59.065639 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:59.065648 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:59.065654 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:59.065940 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:59.066553 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:59.066563 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:59.067064 (+ 501us) client_negotiation.cc:769] Sending connection context
0810 19:32:59.067438 (+ 374us) client_negotiation.cc:240] Negotiation successful
0810 19:32:59.068079 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":335,"threads_started":1}
I20260810 19:32:59.071483 7665 heartbeater.cc:347] Connected to a master server at 127.29.233.190:37269
I20260810 19:32:59.071826 7665 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:59.072567 7665 heartbeater.cc:511] Master 127.29.233.190:37269 requested a full tablet report, sending...
I20260810 19:32:59.074079 7317 ts_manager.cc:194] Registered new tserver with Master: d57c378494074c5a8ca7c6d939f02b3e (127.29.233.130:45997)
I20260810 19:32:59.075047 7317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:44631
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:32:59.114800 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37269
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:32:59.460963 7672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:32:59.461330 7672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:32:59.461488 7672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:32:59.469339 7672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:32:59.469511 7672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:32:59.469694 7672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:32:59.479625 7672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:37269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:32:59.481084 7672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:32:59.482468 7672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:32:59.496527 7678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:59.496625 7677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:32:59.497222 7680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:32:59.497357 7672 server_base.cc:1034] running on GCE node
I20260810 19:32:59.498003 7672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:32:59.498986 7672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:32:59.500181 7672 hybrid_clock.cc:648] HybridClock initialized: now 1786390379500126 us; error 59 us; skew 500 ppm
I20260810 19:32:59.500448 7672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:32:59 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390379, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:32:59.504025 7672 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:32:59.506064 7672 webserver.cc:460] Webserver started at http://127.29.233.131:41605/ using document root <none> and password file <none>
I20260810 19:32:59.507133 7672 fs_manager.cc:359] Metadata directory not provided
I20260810 19:32:59.507294 7672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:32:59.507704 7672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:32:59.512132 7672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "f4d0c07d68444b32b4055781d40b6712"
format_stamp: "Formatted at 2026-08-10 19:32:59 on dist-test-slave-tm1g"
server_key: "849c0c5485fecddc284344d5c8ebdf2e"
server_key_iv: "e735eef657e6a8e7e2401c5e258686d4"
server_key_version: "encryptionkey@0"
I20260810 19:32:59.513281 7672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "f4d0c07d68444b32b4055781d40b6712"
format_stamp: "Formatted at 2026-08-10 19:32:59 on dist-test-slave-tm1g"
server_key: "849c0c5485fecddc284344d5c8ebdf2e"
server_key_iv: "e735eef657e6a8e7e2401c5e258686d4"
server_key_version: "encryptionkey@0"
I20260810 19:32:59.519203 7672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 19:32:59.522838 7687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:59.524169 7672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:32:59.524353 7672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "f4d0c07d68444b32b4055781d40b6712"
format_stamp: "Formatted at 2026-08-10 19:32:59 on dist-test-slave-tm1g"
server_key: "849c0c5485fecddc284344d5c8ebdf2e"
server_key_iv: "e735eef657e6a8e7e2401c5e258686d4"
server_key_version: "encryptionkey@0"
I20260810 19:32:59.524566 7672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:32:59.548382 7672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:32:59.552503 7518 heartbeater.cc:503] Master 127.29.233.190:37269 was elected leader, sending a full tablet report...
I20260810 19:32:59.552940 7672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:32:59.553244 7672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:32:59.554672 7672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:32:59.556073 7672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:32:59.556152 7672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:59.556308 7672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:32:59.556358 7672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:32:59.591876 7672 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:35445
I20260810 19:32:59.591921 7808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:35445 every 8 connection(s)
I20260810 19:32:59.593240 7672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:32:59.603534 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 7672
I20260810 19:32:59.603714 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:32:59.604207 30630 external_mini_cluster.cc:1428] Setting key aeb6267eafd4e7f602696effe2c1f504
Aug 10 19:32:59 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390379, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:32:59.613559 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:59.595821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:52245 (local address 127.29.233.190:37269)
0810 19:32:59.596310 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:59.596322 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:59.597093 (+ 771us) server_negotiation.cc:407] Connection header received
0810 19:32:59.599026 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:59.599032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:59.599122 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:59.599253 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:59.600750 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.601568 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.603203 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.603686 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.603836 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:59.607982 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:59.607987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:59.607990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:59.608032 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:59.610667 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.611440 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.611444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.611446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.611575 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.611993 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.611999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.612003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.612287 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:59.612419 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:59.613087 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 19:32:59.613265 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":263,"threads_started":1}
I20260810 19:32:59.614661 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:59.596213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:52245)
0810 19:32:59.596922 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0810 19:32:59.596967 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 19:32:59.598746 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:32:59.599489 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:32:59.599497 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:32:59.600012 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:32:59.600572 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:59.600590 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.601732 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.601736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:59.603053 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:32:59.603065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.603935 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.603945 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:32:59.604183 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:59.604986 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:32:59.605020 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:32:59.607800 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:32:59.610806 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:59.610812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:59.610833 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:59.611291 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:59.611668 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:32:59.611671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:32:59.611673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:32:59.611855 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:32:59.612417 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:32:59.612422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:32:59.612748 (+ 326us) client_negotiation.cc:769] Sending connection context
0810 19:32:59.612977 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 19:32:59.613400 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":270,"threads_started":1}
I20260810 19:32:59.616189 7809 heartbeater.cc:347] Connected to a master server at 127.29.233.190:37269
I20260810 19:32:59.616465 7809 heartbeater.cc:464] Registering TS with master...
I20260810 19:32:59.617141 7809 heartbeater.cc:511] Master 127.29.233.190:37269 requested a full tablet report, sending...
I20260810 19:32:59.618705 7317 ts_manager.cc:194] Registered new tserver with Master: f4d0c07d68444b32b4055781d40b6712 (127.29.233.131:35445)
I20260810 19:32:59.619722 7317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:52245
I20260810 19:32:59.620306 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab.
Entry for principal oryx/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab.
WARNING: no policy specified for oryx/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab.
Entry for principal oryx/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab.
WARNING: no policy specified for oryx/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab.
Entry for principal oryx/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab.
I20260810 19:32:59.809546 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 7379
Aug 10 19:32:59 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:32:59.827302 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:59.810986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.29.233.129:34519)
0810 19:32:59.811687 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:59.811694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:59.811721 (+ 27us) server_negotiation.cc:407] Connection header received
0810 19:32:59.811783 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:59.811790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:59.812016 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:59.812147 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:59.813750 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.814986 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.816289 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.816886 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.816997 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:59.821598 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:59.821612 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:59.821629 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:59.821670 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:59.824543 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.825220 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.825227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.825229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.825366 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.825709 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.825712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.825714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.826080 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:59.826387 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:59.826747 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 19:32:59.826952 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":378,"threads_started":1}
I20260810 19:32:59.828778 7513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:32:59.917778 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 7379
I20260810 19:32:59.937940 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 7524
Aug 10 19:32:59 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:32:59.954519 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:32:59.939149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.29.233.130:45997)
0810 19:32:59.939755 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 19:32:59.939767 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:32:59.939805 (+ 38us) server_negotiation.cc:407] Connection header received
0810 19:32:59.939874 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:32:59.939880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:32:59.940120 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:32:59.940250 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:32:59.941485 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.942625 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.943884 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:32:59.944433 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:32:59.944526 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:32:59.948866 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:32:59.948877 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:32:59.948892 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:32:59.948925 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:32:59.951787 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.952402 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.952408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.952410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.952535 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:32:59.952919 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:32:59.952922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:32:59.952924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:32:59.953311 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:32:59.953538 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 19:32:59.953879 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 19:32:59.954084 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":293,"threads_started":1}
I20260810 19:32:59.956054 7660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:00.039880 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 7524
I20260810 19:33:00.059844 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 7672
Aug 10 19:33:00 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:33:00.077292 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:00.060917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.29.233.131:35445)
0810 19:33:00.061491 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:00.061498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:00.061529 (+ 31us) server_negotiation.cc:407] Connection header received
0810 19:33:00.061596 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:00.061603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:00.061818 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:00.061964 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:00.063452 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:00.064489 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:00.065889 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:00.066543 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:00.066645 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:00.071150 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:00.071162 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:00.071176 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:00.071212 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:00.074021 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:00.074989 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:00.075004 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:00.075006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:00.075142 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:00.075560 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:00.075564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:00.075567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:00.075966 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:00.076200 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:00.076584 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 19:33:00.076948 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":275,"threads_started":1}
I20260810 19:33:00.078661 7804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:00.163407 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 7672
I20260810 19:33:00.182808 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 7281
I20260810 19:33:00.184075 7348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:00.290107 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 7281
I20260810 19:33:00.314129 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37269
--webserver_interface=127.29.233.190
--webserver_port=45851
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:37269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:33:00.683279 7828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:00.683590 7828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:00.683683 7828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:00.690995 7828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:33:00.691112 7828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:00.691161 7828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:00.691220 7828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:33:00.691265 7828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:33:00.701366 7828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:37269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:37269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=45851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:00.702752 7828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:00.704041 7828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:00.716207 7837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:00.716317 7834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:00.716346 7833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:00.717162 7828 server_base.cc:1034] running on GCE node
I20260810 19:33:00.718008 7828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:00.719519 7828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:00.720752 7828 hybrid_clock.cc:648] HybridClock initialized: now 1786390380720712 us; error 51 us; skew 500 ppm
I20260810 19:33:00.721112 7828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:00 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390380, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:00.724416 7828 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:33:00.725906 7828 webserver.cc:460] Webserver started at http://127.29.233.190:45851/ using document root <none> and password file <none>
I20260810 19:33:00.726785 7828 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:00.726963 7828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:00.732549 7828 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 19:33:00.735491 7844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:00.736794 7828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:33:00.737007 7828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade"
format_stamp: "Formatted at 2026-08-10 19:32:57 on dist-test-slave-tm1g"
server_key: "45b414897687df93d3e259be884bfd8e"
server_key_iv: "06ab30845d3b1087ef16511f92cab4b9"
server_key_version: "encryptionkey@0"
I20260810 19:33:00.738021 7828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:00.760429 7828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:00.765271 7828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:00.765602 7828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:00.785537 7828 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:37269
I20260810 19:33:00.785575 7900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:37269 every 8 connection(s)
I20260810 19:33:00.787195 7828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:33:00.794283 7901 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:33:00.794662 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 7828
I20260810 19:33:00.795501 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:34519
--local_ip_for_outbound_sockets=127.29.233.129
--tserver_master_addrs=127.29.233.190:37269
--webserver_port=39705
--webserver_interface=127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:33:00.797722 7901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Bootstrap starting.
I20260810 19:33:00.817898 7901 log.cc:824] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Log is configured to *not* fsync() on all Append() calls
I20260810 19:33:00.827596 7901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:33:00.828191 7901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Bootstrap complete.
I20260810 19:33:00.837761 7901 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:33:00.838788 7901 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c7510cc4a4ce9a4eb1aecb5762ade, State: Initialized, Role: FOLLOWER
I20260810 19:33:00.839422 7901 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:33:00.839711 7901 raft_consensus.cc:388] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:33:00.839882 7901 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:33:00.840075 7901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 1 FOLLOWER]: Advancing to term 2
I20260810 19:33:00.844571 7901 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:33:00.845115 7901 leader_election.cc:302] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f77c7510cc4a4ce9a4eb1aecb5762ade; no voters:
I20260810 19:33:00.845829 7901 leader_election.cc:288] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 19:33:00.845978 7905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 2 FOLLOWER]: Leader election won for term 2
I20260810 19:33:00.847218 7905 raft_consensus.cc:686] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [term 2 LEADER]: Becoming Leader. State: Replica: f77c7510cc4a4ce9a4eb1aecb5762ade, State: Running, Role: LEADER
I20260810 19:33:00.847903 7905 consensus_queue.cc:237] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } }
I20260810 19:33:00.848254 7901 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:33:00.850663 7906 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } } }
I20260810 19:33:00.851123 7906 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:00.853451 7907 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader f77c7510cc4a4ce9a4eb1aecb5762ade. Latest consensus state: current_term: 2 leader_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77c7510cc4a4ce9a4eb1aecb5762ade" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 37269 } } }
I20260810 19:33:00.853940 7907 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:00.855903 7911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:33:00.864284 7911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:33:00.866720 7911 catalog_manager.cc:1259] Loaded cluster ID: 19ef589069c74fa5829508337df58644
I20260810 19:33:00.866937 7911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:33:00.872939 7911 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:33:00.875005 7911 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f77c7510cc4a4ce9a4eb1aecb5762ade: Loaded TSK: 0
I20260810 19:33:00.878564 7911 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 19:33:01.163705 7903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:01.164080 7903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:01.164268 7903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:01.171752 7903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:01.171916 7903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:01.172096 7903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:33:01.181947 7903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.129.keytab
--principal=oryx/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:34519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=39705
--tserver_master_addrs=127.29.233.190:37269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:01.183583 7903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:01.184934 7903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:01.200317 7928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:01.200412 7931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:01.200958 7927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:01.200932 7903 server_base.cc:1034] running on GCE node
I20260810 19:33:01.201818 7903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:01.202953 7903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:01.204250 7903 hybrid_clock.cc:648] HybridClock initialized: now 1786390381204191 us; error 48 us; skew 500 ppm
I20260810 19:33:01.204656 7903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390381, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:01.208341 7903 init.cc:377] Logged in from keytab as oryx/127.29.233.129@KRBTEST.COM (short username oryx)
I20260810 19:33:01.210265 7903 webserver.cc:460] Webserver started at http://127.29.233.129:39705/ using document root <none> and password file <none>
I20260810 19:33:01.211102 7903 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:01.211256 7903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:01.217255 7903 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 19:33:01.220219 7940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:01.221661 7903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:33:01.221859 7903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "fc6c302390f94495bbe9b29dd2b807d5"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "add81d70ca6538bb85d66f55ebb9c37a"
server_key_iv: "532bd3bf6375c8a08f030e254978ad71"
server_key_version: "encryptionkey@0"
I20260810 19:33:01.222983 7903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:01.243709 7903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:01.248255 7903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:01.248565 7903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:01.249787 7903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:01.251256 7903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:01.251344 7903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:01.251473 7903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:01.251579 7903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:01.287909 7903 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:34519
I20260810 19:33:01.288002 8063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:34519 every 8 connection(s)
I20260810 19:33:01.289431 7903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:33:01.297231 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 7903
I20260810 19:33:01.298054 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:45997
--local_ip_for_outbound_sockets=127.29.233.130
--tserver_master_addrs=127.29.233.190:37269
--webserver_port=36851
--webserver_interface=127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:01.310710 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:01.292602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:53515)
0810 19:33:01.293441 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:01.293478 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:01.295217 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:01.296627 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:01.296640 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:01.297457 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:01.298260 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.298329 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.300030 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.300038 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.302162 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.302182 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.303349 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.303360 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.303684 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.304451 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:01.304492 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:01.309190 (+ 4698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":392,"threads_started":1}
W20260810 19:33:01.310742 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:01.292586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53515 (local address 127.29.233.190:37269)
0810 19:33:01.293370 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:01.293385 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:01.293616 (+ 231us) server_negotiation.cc:407] Connection header received
0810 19:33:01.295738 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:01.295750 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:01.296063 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:01.296358 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:01.298525 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.299845 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.302401 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.303069 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.303217 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.309611 (+ 6394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:53515: BlockingRecv error: recv got EOF from 127.29.233.129:53515 (error 108)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":301,"threads_started":1}
W20260810 19:33:01.311823 8064 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:37269 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.233.190:37269: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:01.321583 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:01.312626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:41083)
0810 19:33:01.312975 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:01.312996 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:01.313113 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:01.313526 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:01.313531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:01.313914 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:01.314063 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.314072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.315297 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.315304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.316307 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.316321 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.317109 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.317116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.317305 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.317972 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:01.317991 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:01.321249 (+ 3258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185}
W20260810 19:33:01.321764 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:01.312978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:41083 (local address 127.29.233.190:37269)
0810 19:33:01.313212 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:01.313218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:01.313236 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:33:01.313301 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:01.313313 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:01.313374 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:01.313495 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:01.314220 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.315138 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.316472 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.316936 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.317082 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.321509 (+ 4427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:41083: BlockingRecv error: recv got EOF from 127.29.233.129:41083 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:01.333053 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:01.322675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:43079)
0810 19:33:01.322905 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:01.322929 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:01.323053 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:01.323653 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:01.323661 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:01.324140 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:01.324382 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.324399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.325579 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.325587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.327449 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.327469 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.328290 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.328296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.328552 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.329241 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:01.329264 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:01.332728 (+ 3464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 19:33:01.333536 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:01.322841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:43079 (local address 127.29.233.190:37269)
0810 19:33:01.323009 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:01.323013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:01.323031 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:33:01.323360 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:01.323366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:01.323485 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:01.323581 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:01.324574 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.325407 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.327610 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.328120 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.328228 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.332991 (+ 4763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:43079: BlockingRecv error: recv got EOF from 127.29.233.129:43079 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 19:33:01.333509 8064 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 19:33:01.650441 8068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:01.650766 8068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:01.650960 8068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:01.658473 8068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:01.658620 8068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:01.658811 8068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:33:01.668922 8068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.130.keytab
--principal=oryx/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:45997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=36851
--tserver_master_addrs=127.29.233.190:37269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:01.670382 8068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:01.671713 8068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:01.685431 8073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:01.685477 8074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:01.685441 8076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:01.686358 8068 server_base.cc:1034] running on GCE node
I20260810 19:33:01.687008 8068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:01.688045 8068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:01.689313 8068 hybrid_clock.cc:648] HybridClock initialized: now 1786390381689254 us; error 47 us; skew 500 ppm
I20260810 19:33:01.689713 8068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390381, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:01.693195 8068 init.cc:377] Logged in from keytab as oryx/127.29.233.130@KRBTEST.COM (short username oryx)
I20260810 19:33:01.694895 8068 webserver.cc:460] Webserver started at http://127.29.233.130:36851/ using document root <none> and password file <none>
I20260810 19:33:01.695684 8068 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:01.695852 8068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:01.701782 8068 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 19:33:01.704721 8084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:01.706140 8068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:33:01.706400 8068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "d57c378494074c5a8ca7c6d939f02b3e"
format_stamp: "Formatted at 2026-08-10 19:32:58 on dist-test-slave-tm1g"
server_key: "398ba8438c769d4b88c17a7738c998f6"
server_key_iv: "674b8130243848033963fc22c28ce69b"
server_key_version: "encryptionkey@0"
I20260810 19:33:01.707583 8068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:01.744776 8068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:01.749892 8068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:01.750144 8068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:01.751209 8068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:01.752635 8068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:01.752722 8068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:01.752888 8068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:01.752967 8068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:01.791225 8068 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:45997
I20260810 19:33:01.791280 8206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:45997 every 8 connection(s)
I20260810 19:33:01.792793 8068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:33:01.800236 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 8068
I20260810 19:33:01.801107 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:35445
--local_ip_for_outbound_sockets=127.29.233.131
--tserver_master_addrs=127.29.233.190:37269
--webserver_port=41605
--webserver_interface=127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 19:33:01.810665 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:01.795731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:54107 (local address 127.29.233.190:37269)
0810 19:33:01.795925 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:01.795929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:01.797171 (+ 1242us) server_negotiation.cc:407] Connection header received
0810 19:33:01.799090 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:01.799095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:01.799164 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:01.799258 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:01.800817 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.801926 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.803545 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.804266 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.804421 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.810246 (+ 5825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:54107: BlockingRecv error: recv got EOF from 127.29.233.130:54107 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:33:01.811628 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:01.796234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:54107)
0810 19:33:01.797009 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:01.797047 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:01.798798 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:01.799487 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:01.799497 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:01.800008 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:01.800640 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.800658 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.802103 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.802107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.803391 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.803403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.804490 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.804495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.804689 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.805401 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:01.805423 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:01.810022 (+ 4599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":313,"threads_started":1}
W20260810 19:33:01.812322 8208 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:37269 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.233.190:37269: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:01.821601 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:01.813006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:49023)
0810 19:33:01.813240 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:01.813263 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:01.813385 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:01.813720 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:01.813725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:01.814090 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:01.814235 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.814243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.815356 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.815361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.816272 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.816284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.817039 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.817045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.817224 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.817846 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:01.817864 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:01.821241 (+ 3377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260810 19:33:01.822165 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:01.813191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:49023 (local address 127.29.233.190:37269)
0810 19:33:01.813363 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:01.813367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:01.813385 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:33:01.813503 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:01.813507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:01.813568 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:01.813690 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:01.814436 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.815199 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.816426 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.816878 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.817025 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.821504 (+ 4479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:49023: BlockingRecv error: recv got EOF from 127.29.233.130:49023 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:01 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 19:33:01.830932 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:01.822642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:43645 (local address 127.29.233.190:37269)
0810 19:33:01.822855 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:01.822861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:01.822885 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:33:01.822930 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:01.822933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:01.822990 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:01.823098 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:01.823749 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.824616 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.825888 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.826464 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.826605 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.830752 (+ 4147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:43645: BlockingRecv error: recv got EOF from 127.29.233.130:43645 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:33:01.830981 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:01.822414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:43645)
0810 19:33:01.822607 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:01.822625 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:01.822746 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:01.823125 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:01.823130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:01.823449 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:01.823600 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.823609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.824754 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.824766 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.825723 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:01.825734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:01.826615 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:01.826621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:01.826880 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:01.827497 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:01.827515 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:01.830578 (+ 3063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 19:33:01.831375 8208 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 19:33:02.156714 8211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:02.157013 8211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:02.157109 8211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:02.164681 8211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:02.164810 8211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:02.164928 8211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:33:02.176131 8211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/krb5kdc/oryx_127.29.233.131.keytab
--principal=oryx/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:35445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=41605
--tserver_master_addrs=127.29.233.190:37269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:02.177531 8211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:02.178921 8211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:02.192773 8216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:02.192873 8217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:02.192826 8219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:02.193907 8211 server_base.cc:1034] running on GCE node
I20260810 19:33:02.194706 8211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:02.195722 8211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:02.197104 8211 hybrid_clock.cc:648] HybridClock initialized: now 1786390382197047 us; error 75 us; skew 500 ppm
I20260810 19:33:02.197482 8211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390382, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:02.200901 8211 init.cc:377] Logged in from keytab as oryx/127.29.233.131@KRBTEST.COM (short username oryx)
I20260810 19:33:02.202490 8211 webserver.cc:460] Webserver started at http://127.29.233.131:41605/ using document root <none> and password file <none>
I20260810 19:33:02.203192 8211 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:02.203315 8211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:02.210014 8211 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 19:33:02.213248 8226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:02.214761 8211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 19:33:02.214906 8211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "f4d0c07d68444b32b4055781d40b6712"
format_stamp: "Formatted at 2026-08-10 19:32:59 on dist-test-slave-tm1g"
server_key: "849c0c5485fecddc284344d5c8ebdf2e"
server_key_iv: "e735eef657e6a8e7e2401c5e258686d4"
server_key_version: "encryptionkey@0"
I20260810 19:33:02.215919 8211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:02.233232 8211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:02.238265 8211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:02.239089 8211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:02.240232 8211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:02.241938 8211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:02.242076 8211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:02.242230 8211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:02.242362 8211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:02.282284 8211 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:35445
I20260810 19:33:02.282363 8351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:35445 every 8 connection(s)
I20260810 19:33:02.283957 8211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:33:02.284507 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 8211
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 19:33:02.303105 8355 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:02.287041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:51747 (local address 127.29.233.190:37269)
0810 19:33:02.289699 (+ 2658us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:02.289705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:02.289723 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:33:02.291096 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:02.291102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:02.291189 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:02.291304 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:02.293079 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.294294 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.296010 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.296489 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.296638 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.302744 (+ 6106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:51747: BlockingRecv error: recv got EOF from 127.29.233.131:51747 (error 108)
Metrics: {"server-negotiator.queue_time_us":2551,"thread_start_us":2699,"threads_started":1}
I20260810 19:33:02.303839 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:02.287617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:51747)
0810 19:33:02.288484 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:02.288539 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:02.290833 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:02.291586 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:02.291599 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:02.292123 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:02.292850 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.292877 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.294489 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.294493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.295858 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.295872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.296740 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.296749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.297024 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.297763 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:02.297786 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:02.302285 (+ 4499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":372,"threads_started":1}
I20260810 19:33:02.304137 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:02.285677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.29.233.190:37269)
0810 19:33:02.285936 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:02.285943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:02.286777 (+ 834us) server_negotiation.cc:407] Connection header received
0810 19:33:02.287050 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:02.287057 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:02.287150 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:02.287269 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:02.289407 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.290628 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.292511 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.293151 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.293327 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.296516 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:02.296526 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:02.296542 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:02.296582 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:02.300058 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:02.300921 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:02.300932 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:02.300941 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:02.301192 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:02.301682 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:02.301689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:02.301693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:02.302474 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:02.302843 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:02.303345 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 19:33:02.303743 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 19:33:02.304551 8352 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:37269 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.233.190:37269: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:02.316095 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:02.305210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:47651)
0810 19:33:02.305536 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:02.305694 (+ 158us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:02.305835 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:02.306474 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:02.306478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:02.307003 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:02.307188 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.307197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.308830 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.308838 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.310196 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.310217 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.311168 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.311173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.311322 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.312182 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:02.312202 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:02.315879 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 19:33:02.316628 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:02.305762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:47651 (local address 127.29.233.190:37269)
0810 19:33:02.306001 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:02.306007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:02.306025 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:33:02.306086 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:02.306091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:02.306198 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:02.306330 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:02.307357 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.308457 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.310488 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.310981 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.311142 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.316125 (+ 4983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:47651: BlockingRecv error: recv got EOF from 127.29.233.131:47651 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:02.326560 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:02.316907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:45067)
0810 19:33:02.317114 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:02.317132 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:02.317226 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:02.317703 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:02.317708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:02.318044 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:02.318199 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.318208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.319290 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.319297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.320538 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.320550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.321489 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.321504 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.321713 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.322650 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:02.322682 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:02.326052 (+ 3370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 19:33:02.326566 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:02.317188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:45067 (local address 127.29.233.190:37269)
0810 19:33:02.317437 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:02.317442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:02.317456 (+ 14us) server_negotiation.cc:407] Connection header received
0810 19:33:02.317502 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:02.317507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:02.317564 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:02.317676 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:02.318368 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.319139 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.320745 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.321306 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.321479 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.326274 (+ 4795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:45067: BlockingRecv error: recv got EOF from 127.29.233.131:45067 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
W20260810 19:33:02.327004 8352 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:02.346292 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:02.334681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:37193)
0810 19:33:02.335297 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:02.335315 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:02.335443 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:02.335884 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:02.335889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:02.336384 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:02.336598 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.336608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.337851 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.337859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.339341 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.339361 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.340743 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.340752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.340951 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.341911 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:02.341949 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:02.346028 (+ 4079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":339,"threads_started":1}
W20260810 19:33:02.346730 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:02.334735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:37193 (local address 127.29.233.190:37269)
0810 19:33:02.334973 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:02.334980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:02.335403 (+ 423us) server_negotiation.cc:407] Connection header received
0810 19:33:02.335556 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:02.335561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:02.335643 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:02.335771 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:02.336755 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.337542 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.339591 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.340329 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.340509 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.346459 (+ 5950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:37193: BlockingRecv error: recv got EOF from 127.29.233.129:37193 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:02 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:02.847218 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:02.832183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:54565)
0810 19:33:02.832782 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:02.832800 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:02.832988 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:02.833369 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:02.833374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:02.834055 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:02.834285 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.834295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.835635 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.835640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.836631 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:02.836643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.837458 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.837465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:02.837640 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.843459 (+ 5819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:02.843486 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:02.846956 (+ 3470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":332,"threads_started":1}
W20260810 19:33:02.847657 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:02.832311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:54565 (local address 127.29.233.190:37269)
0810 19:33:02.832608 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:02.832617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:02.832892 (+ 275us) server_negotiation.cc:407] Connection header received
0810 19:33:02.833127 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:02.833138 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:02.833206 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:02.833334 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:02.834520 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.835481 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.836792 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:02.837255 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:02.837416 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:02.847359 (+ 9943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:54565: BlockingRecv error: recv got EOF from 127.29.233.130:54565 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 10 19:33:03 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:03 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:03.337106 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:03.327674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:32939)
0810 19:33:03.328215 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:03.328233 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:03.328368 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:03.328792 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:03.328797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:03.329348 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:03.329550 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:03.329560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.330725 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.330730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:03.331752 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:03.331764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.332734 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.332739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:03.332948 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:03.333693 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:03.333711 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:03.336869 (+ 3158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":306,"threads_started":1}
W20260810 19:33:03.337334 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:03.327772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:32939 (local address 127.29.233.190:37269)
0810 19:33:03.328065 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:03.328072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:03.328344 (+ 272us) server_negotiation.cc:407] Connection header received
0810 19:33:03.328536 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:03.328542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:03.328609 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:03.328700 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:03.329704 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.330564 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.331897 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.332543 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.332672 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:03.337093 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:32939: BlockingRecv error: recv got EOF from 127.29.233.131:32939 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 10 19:33:03 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:03 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:03.355796 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:03.347460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:60985)
0810 19:33:03.348075 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:03.348091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:03.348247 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:03.348586 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:03.348589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:03.348996 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:03.349223 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:03.349254 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.350262 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.350266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:03.351232 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:03.351244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.351955 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.351959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:03.352064 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:03.352723 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:03.352741 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:03.355550 (+ 2809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":321,"threads_started":1}
W20260810 19:33:03.356072 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:03.347500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:60985 (local address 127.29.233.190:37269)
0810 19:33:03.347756 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:03.347762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:03.348188 (+ 426us) server_negotiation.cc:407] Connection header received
0810 19:33:03.348370 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:03.348374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:03.348459 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:03.348546 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:03.349404 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.350141 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.351378 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.351808 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.351904 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:03.355778 (+ 3874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:60985: BlockingRecv error: recv got EOF from 127.29.233.129:60985 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 19:33:03 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:03 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 19:33:03.857664 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:03.848374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:59989 (local address 127.29.233.190:37269)
0810 19:33:03.848624 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:03.848633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:03.849117 (+ 484us) server_negotiation.cc:407] Connection header received
0810 19:33:03.849353 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:03.849421 (+ 68us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:03.849525 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:03.849645 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:03.850511 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.851249 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.852498 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.852947 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.853087 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:03.857403 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:59989: BlockingRecv error: recv got EOF from 127.29.233.130:59989 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:33:03.858196 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:03.848243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:59989)
0810 19:33:03.848997 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:03.849024 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:03.849191 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:03.849682 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:03.849687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:03.850135 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:03.850359 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:03.850374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.851396 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.851402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:03.852352 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:03.852367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:03.853105 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:03.853111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:03.853282 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:03.853963 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:03.853981 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:03.857889 (+ 3908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":395,"threads_started":1}
Aug 10 19:33:04 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:04 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:04.347568 8363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:04.338228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:38871)
0810 19:33:04.338868 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:04.338886 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:04.339015 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:04.339365 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:04.339369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:04.339756 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:04.340052 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:04.340064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.341479 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.341484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:04.342572 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:04.342593 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.343417 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.343422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:04.343544 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:04.344173 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:04.344192 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:04.347274 (+ 3082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":365,"threads_started":1}
W20260810 19:33:04.347960 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:04.338528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:38871 (local address 127.29.233.190:37269)
0810 19:33:04.338749 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:04.338756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:04.338994 (+ 238us) server_negotiation.cc:407] Connection header received
0810 19:33:04.339127 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:04.339131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:04.339204 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:04.339296 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:04.340206 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.341324 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.342747 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.343199 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.343367 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:04.347536 (+ 4169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:38871: BlockingRecv error: recv got EOF from 127.29.233.131:38871 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 19:33:04 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:04 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:04.365729 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:04.356783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:41991)
0810 19:33:04.357356 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:04.357373 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:04.357567 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:04.357939 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:04.357943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:04.358367 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:04.358565 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:04.358575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.359764 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.359768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:04.360699 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:04.360710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.361478 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.361482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:04.361610 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:04.362204 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:04.362221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:04.365330 (+ 3109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":306,"threads_started":1}
W20260810 19:33:04.365901 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:04.356931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:41991 (local address 127.29.233.190:37269)
0810 19:33:04.357177 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:04.357186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:04.357494 (+ 308us) server_negotiation.cc:407] Connection header received
0810 19:33:04.357728 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:04.357736 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:04.357806 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:04.357929 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:04.358763 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.359613 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.360858 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.361298 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.361440 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:04.365578 (+ 4138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:41991: BlockingRecv error: recv got EOF from 127.29.233.129:41991 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 10 19:33:04 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:04 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:04.868263 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:04.859264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:48389)
0810 19:33:04.859834 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:04.859852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:04.859983 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:04.860379 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:04.860383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:04.860740 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:04.860921 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:04.860930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.861995 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.861999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:04.863026 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:04.863039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.863748 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.863753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:04.863901 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:04.864577 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:04.864595 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:04.867927 (+ 3332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":331,"threads_started":1}
W20260810 19:33:04.868352 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:04.859424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:48389 (local address 127.29.233.190:37269)
0810 19:33:04.859597 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:04.859602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:04.859929 (+ 327us) server_negotiation.cc:407] Connection header received
0810 19:33:04.860083 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:04.860087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:04.860187 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:04.860282 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:04.861084 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.861864 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.863159 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:04.863595 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:04.863693 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:04.868132 (+ 4439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:48389: BlockingRecv error: recv got EOF from 127.29.233.130:48389 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 19:33:05 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:05 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:05.357203 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:05.348616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:35789)
0810 19:33:05.349097 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:05.349115 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:05.349236 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:05.349592 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:05.349596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:05.349965 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:05.350169 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:05.350178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.351260 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.351272 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:05.352183 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:05.352195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.352921 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.352926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:05.353123 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:05.353763 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:05.353782 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:05.356810 (+ 3028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":266,"threads_started":1}
W20260810 19:33:05.357303 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:05.348710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:35789 (local address 127.29.233.190:37269)
0810 19:33:05.348954 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:05.348961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:05.349216 (+ 255us) server_negotiation.cc:407] Connection header received
0810 19:33:05.349341 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:05.349345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:05.349445 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:05.349536 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:05.350362 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.351109 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.352325 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.352764 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.352897 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:05.357057 (+ 4160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:35789: BlockingRecv error: recv got EOF from 127.29.233.131:35789 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 19:33:05 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:05 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:05.375094 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:05.366651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:58509)
0810 19:33:05.367200 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:05.367216 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:05.367377 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:05.367723 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:05.367727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:05.368123 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:05.368275 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:05.368284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.369319 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.369326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:05.370397 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:05.370410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.371196 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.371202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:05.371373 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:05.371994 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:05.372021 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:05.374867 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":315,"threads_started":1}
W20260810 19:33:05.375339 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:05.366733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:58509 (local address 127.29.233.190:37269)
0810 19:33:05.367026 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:05.367035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:05.367304 (+ 269us) server_negotiation.cc:407] Connection header received
0810 19:33:05.367511 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:05.367516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:05.367590 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:05.367693 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:05.368426 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.369186 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.370537 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.371038 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.371178 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:05.375075 (+ 3897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:58509: BlockingRecv error: recv got EOF from 127.29.233.129:58509 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 10 19:33:05 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:05 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:05.877925 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:05.869267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:43899)
0810 19:33:05.869827 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:05.869843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:05.870013 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:05.870423 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:05.870428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:05.870831 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:05.870981 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:05.870989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.872018 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.872025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:05.873001 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:05.873012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.873731 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.873739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:05.873921 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:05.874572 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:05.874590 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:05.877666 (+ 3076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":322,"threads_started":1}
W20260810 19:33:05.878132 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:05.869370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:43899 (local address 127.29.233.190:37269)
0810 19:33:05.869627 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:05.869636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:05.869954 (+ 318us) server_negotiation.cc:407] Connection header received
0810 19:33:05.870145 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:05.870157 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:05.870236 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:05.870386 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:05.871132 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.871872 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.873149 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:05.873577 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:05.873718 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:05.877894 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:43899: BlockingRecv error: recv got EOF from 127.29.233.130:43899 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 10 19:33:06 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:06 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:06.367287 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:06.358368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:47369)
0810 19:33:06.358935 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:06.358962 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:06.359122 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:06.359517 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:06.359522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:06.359945 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:06.360095 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:06.360104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.361163 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.361170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:06.362187 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:06.362199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.362971 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.362977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:06.363153 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:06.363763 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:06.363781 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:06.366955 (+ 3174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":289,"threads_started":1}
W20260810 19:33:06.367460 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:06.358426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:47369 (local address 127.29.233.190:37269)
0810 19:33:06.358666 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:06.358674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:06.359054 (+ 380us) server_negotiation.cc:407] Connection header received
0810 19:33:06.359270 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:06.359278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:06.359365 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:06.359483 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:06.360248 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.361022 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.362362 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.362805 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.362953 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:06.367188 (+ 4235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:47369: BlockingRecv error: recv got EOF from 127.29.233.131:47369 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 19:33:06 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:06 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:06.384702 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:06.376081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:43177)
0810 19:33:06.376606 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:06.376623 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:06.376736 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:06.377111 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:06.377115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:06.377678 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:06.377877 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:06.377887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.379172 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.379176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:06.380089 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:06.380100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.380746 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.380750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:06.380883 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:06.381497 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:06.381514 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:06.384455 (+ 2941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":255,"threads_started":1}
W20260810 19:33:06.384886 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:06.376154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:43177 (local address 127.29.233.190:37269)
0810 19:33:06.376387 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:06.376393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:06.376712 (+ 319us) server_negotiation.cc:407] Connection header received
0810 19:33:06.376850 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:06.376856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:06.376961 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:06.377045 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:06.378040 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.379050 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.380212 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.380624 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.380717 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:06.384653 (+ 3936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:43177: BlockingRecv error: recv got EOF from 127.29.233.129:43177 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 19:33:06 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:06 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:06.888515 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:06.879123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:51671)
0810 19:33:06.879712 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:06.879730 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:06.879861 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:06.880231 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:06.880235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:06.880656 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:06.880846 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:06.880855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.882335 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.882341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:06.883411 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:06.883424 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.884198 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.884204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:06.884394 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:06.885037 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:06.885055 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:06.888236 (+ 3181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":294,"threads_started":1}
W20260810 19:33:06.888749 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:06.879442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:51671 (local address 127.29.233.190:37269)
0810 19:33:06.879688 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:06.879693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:06.879817 (+ 124us) server_negotiation.cc:407] Connection header received
0810 19:33:06.879984 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:06.879989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:06.880066 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:06.880155 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:06.881031 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.882141 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.883556 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:06.884023 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:06.884172 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:06.888504 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:51671: BlockingRecv error: recv got EOF from 127.29.233.130:51671 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 10 19:33:07 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:07 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:07.377795 8373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:07.368505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:38873)
0810 19:33:07.369375 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:07.369392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:07.369505 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:07.370019 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:07.370023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:07.370781 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:07.370952 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:07.370961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.372005 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.372009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:07.372976 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:07.372988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.373783 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.373788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:07.373932 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:07.374671 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:07.374689 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:07.377544 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":601,"threads_started":1}
W20260810 19:33:07.378084 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:07.368645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:38873 (local address 127.29.233.190:37269)
0810 19:33:07.368839 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:07.368851 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:07.369467 (+ 616us) server_negotiation.cc:407] Connection header received
0810 19:33:07.369637 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:07.369642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:07.369858 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:07.369988 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:07.371087 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.371859 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.373130 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.373609 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.373720 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:07.377774 (+ 4054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:38873: BlockingRecv error: recv got EOF from 127.29.233.131:38873 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 19:33:07 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:07 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:07.395113 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:07.385752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:54699)
0810 19:33:07.386404 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:07.386421 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:07.386536 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:07.386884 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:07.386888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:07.387274 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:07.387465 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:07.387474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.388560 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.388564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:07.389950 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:07.389964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.390807 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.390811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:07.390934 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:07.391517 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:07.391535 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:07.394774 (+ 3239us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":347,"threads_started":1}
W20260810 19:33:07.395293 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:07.385910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:54699 (local address 127.29.233.190:37269)
0810 19:33:07.386092 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:07.386097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:07.386500 (+ 403us) server_negotiation.cc:407] Connection header received
0810 19:33:07.386637 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:07.386642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:07.386710 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:07.386806 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:07.387628 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.388393 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.390099 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.390637 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.390741 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:07.395014 (+ 4273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:54699: BlockingRecv error: recv got EOF from 127.29.233.129:54699 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 19:33:07 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:07 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:07.904644 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:07.889581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:40791)
0810 19:33:07.890188 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:07.890205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:07.890407 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:07.890763 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:07.890768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:07.891196 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:07.891357 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:07.891367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.892384 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.892390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:07.893511 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:07.893524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.894275 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.894283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:07.894497 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:07.900653 (+ 6156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:07.900682 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:07.904264 (+ 3582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":402,"threads_started":1}
W20260810 19:33:07.904795 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:07.889742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:40791 (local address 127.29.233.190:37269)
0810 19:33:07.889931 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:07.889939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:07.890335 (+ 396us) server_negotiation.cc:407] Connection header received
0810 19:33:07.890552 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:07.890556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:07.890621 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:07.890728 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:07.891502 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.892240 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.893650 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:07.894099 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:07.894235 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:07.904519 (+ 10284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:40791: BlockingRecv error: recv got EOF from 127.29.233.130:40791 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 19:33:08 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:08 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:08.387974 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:08.379014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:34501)
0810 19:33:08.379585 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:08.379601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:08.379783 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:08.380163 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:08.380168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:08.380624 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:08.380779 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:08.380787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.381877 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.381883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:08.382841 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:08.382852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.383646 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.383653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:08.383827 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:08.384520 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:08.384539 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:08.387687 (+ 3148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":287,"threads_started":1}
W20260810 19:33:08.388139 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:08.379246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:34501 (local address 127.29.233.190:37269)
0810 19:33:08.379464 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:08.379470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:08.379707 (+ 237us) server_negotiation.cc:407] Connection header received
0810 19:33:08.379929 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:08.379934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:08.380016 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:08.380130 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:08.380932 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.381726 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.383002 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.383469 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.383609 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:08.387925 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:34501: BlockingRecv error: recv got EOF from 127.29.233.131:34501 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 10 19:33:08 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:08 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:08.406088 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:08.396347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:58999)
0810 19:33:08.396939 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:08.396957 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:08.397135 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:08.397508 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:08.397513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:08.398029 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:08.398213 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:08.398223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.399388 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.399394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:08.400435 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:08.400448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.401266 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.401273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:08.401456 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:08.402077 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:08.402096 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:08.405741 (+ 3645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":242,"threads_started":1}
W20260810 19:33:08.406538 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:08.396417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:58999 (local address 127.29.233.190:37269)
0810 19:33:08.396667 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:08.396676 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:08.397055 (+ 379us) server_negotiation.cc:407] Connection header received
0810 19:33:08.397270 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:08.397276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:08.397360 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:08.397479 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:08.398397 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.399219 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.400592 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.401076 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.401240 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:08.406190 (+ 4950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:58999: BlockingRecv error: recv got EOF from 127.29.233.129:58999 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 10 19:33:08 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:08 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:08.915752 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:08.905825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:57859)
0810 19:33:08.906652 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:08.906677 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:08.906879 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:08.907264 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:08.907269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:08.907799 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:08.907981 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:08.907991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.909160 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.909166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:08.910206 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:08.910218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.911182 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.911191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:08.911378 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:08.912081 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:08.912100 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:08.915507 (+ 3407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":352,"threads_started":1}
W20260810 19:33:08.916008 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:08.905967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:57859 (local address 127.29.233.190:37269)
0810 19:33:08.906270 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:08.906280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:08.906793 (+ 513us) server_negotiation.cc:407] Connection header received
0810 19:33:08.907026 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:08.907031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:08.907100 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:08.907222 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:08.908148 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.908994 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.910492 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:08.910945 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:08.911095 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:08.915750 (+ 4655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:57859: BlockingRecv error: recv got EOF from 127.29.233.130:57859 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 10 19:33:09 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:09 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:09.398146 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:09.389389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:35105)
0810 19:33:09.390066 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:09.390083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:09.390203 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:09.390514 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:09.390517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:09.390950 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:09.391116 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:09.391124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.392268 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.392272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:09.393223 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:09.393234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.394016 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.394020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:09.394134 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:09.394849 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:09.394868 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:09.397902 (+ 3034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":310,"threads_started":1}
W20260810 19:33:09.398412 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:09.389442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:35105 (local address 127.29.233.190:37269)
0810 19:33:09.389676 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:09.389689 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:09.390185 (+ 496us) server_negotiation.cc:407] Connection header received
0810 19:33:09.390316 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:09.390320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:09.390386 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:09.390474 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:09.391291 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.392115 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.393397 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.393862 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.393974 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:09.398123 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:35105: BlockingRecv error: recv got EOF from 127.29.233.131:35105 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 19:33:09 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:09 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:09.416636 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:09.407365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:41697)
0810 19:33:09.407990 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:09.408006 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:09.408120 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:09.408398 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:09.408402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:09.408763 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:09.408916 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:09.408925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.409930 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.409934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:09.411088 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:09.411102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.411962 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.411968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:09.412143 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:09.412900 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:09.412924 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:09.416382 (+ 3458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":338,"threads_started":1}
W20260810 19:33:09.416812 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:09.407554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:41697 (local address 127.29.233.190:37269)
0810 19:33:09.407721 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:09.407725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:09.408080 (+ 355us) server_negotiation.cc:407] Connection header received
0810 19:33:09.408213 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:09.408217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:09.408277 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:09.408371 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:09.409060 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.409796 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.411240 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.411759 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.411883 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:09.416599 (+ 4716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:41697: BlockingRecv error: recv got EOF from 127.29.233.129:41697 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 19:33:09 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:09 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:09.926343 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:09.916851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:54193)
0810 19:33:09.917508 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:09.917527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:09.917657 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:09.917966 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:09.917969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:09.918457 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:09.918625 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:09.918634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.920193 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.920199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:09.921299 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:09.921312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.922066 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.922074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:09.922250 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:09.922991 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:09.923010 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:09.925945 (+ 2935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":325,"threads_started":1}
W20260810 19:33:09.926463 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:09.917092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:54193 (local address 127.29.233.190:37269)
0810 19:33:09.917329 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:09.917335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:09.917618 (+ 283us) server_negotiation.cc:407] Connection header received
0810 19:33:09.917765 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:09.917769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:09.917834 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:09.917920 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:09.918799 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.920021 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.921449 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:09.921878 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:09.922033 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:09.926194 (+ 4161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:54193: BlockingRecv error: recv got EOF from 127.29.233.130:54193 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 19:33:10 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:10 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:10.408603 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:10.399262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:50259)
0810 19:33:10.399922 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:10.399939 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:10.400056 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:10.400361 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:10.400365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:10.400775 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:10.400927 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:10.400935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.401944 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.401948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:10.403062 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:10.403074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.404425 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.404434 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:10.404605 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:10.405299 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:10.405325 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:10.408329 (+ 3004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":396,"threads_started":1}
W20260810 19:33:10.408782 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:10.399399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:50259 (local address 127.29.233.190:37269)
0810 19:33:10.399602 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:10.399606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:10.400020 (+ 414us) server_negotiation.cc:407] Connection header received
0810 19:33:10.400162 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:10.400167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:10.400231 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:10.400317 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:10.401069 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.401799 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.403234 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.404035 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.404211 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:10.408550 (+ 4339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:50259: BlockingRecv error: recv got EOF from 127.29.233.131:50259 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 19:33:10 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:10 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:10.426375 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:10.417770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:52571)
0810 19:33:10.418393 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:10.418412 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:10.418538 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:10.418874 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:10.418877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:10.419282 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:10.419451 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:10.419459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.420495 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.420499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:10.421605 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:10.421618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.422393 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.422398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:10.422541 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:10.423189 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:10.423207 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:10.426071 (+ 2864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":298,"threads_started":1}
W20260810 19:33:10.426676 8066 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:10.417846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:52571 (local address 127.29.233.190:37269)
0810 19:33:10.418088 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:10.418096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:10.418491 (+ 395us) server_negotiation.cc:407] Connection header received
0810 19:33:10.418636 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:10.418640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:10.418718 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:10.418802 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:10.419601 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.420362 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.421738 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.422183 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.422279 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:10.426273 (+ 3994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:52571: BlockingRecv error: recv got EOF from 127.29.233.129:52571 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 19:33:10 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:10 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:10.936837 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:10.927439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:43679)
0810 19:33:10.928645 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:10.928663 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:10.928781 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:10.929134 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:10.929137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:10.929569 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:10.929745 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:10.929754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.930835 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.930839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:10.931806 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:10.931818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.932584 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.932591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:10.932768 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:10.933357 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:10.933374 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:10.936514 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1099,"thread_start_us":920,"threads_started":1}
W20260810 19:33:10.937140 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:10.927532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:43679 (local address 127.29.233.190:37269)
0810 19:33:10.928204 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:10.928208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:10.928762 (+ 554us) server_negotiation.cc:407] Connection header received
0810 19:33:10.928883 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:10.928887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:10.928953 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:10.929054 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:10.929880 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.930707 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.931975 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:10.932424 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:10.932564 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:10.936810 (+ 4246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:43679: BlockingRecv error: recv got EOF from 127.29.233.130:43679 (error 108)
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":366,"threads_started":1}
Aug 10 19:33:11 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:11 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:11.419764 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:11.409614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:43675)
0810 19:33:11.410197 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:11.410223 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:11.410431 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:11.410938 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:11.410946 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:11.411485 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:11.411763 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:11.411780 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.413517 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.413522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:11.414543 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:11.414555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.415267 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.415273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:11.415455 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:11.416163 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:11.416180 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:11.419371 (+ 3191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":348,"threads_started":1}
W20260810 19:33:11.419869 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:11.409718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:43675 (local address 127.29.233.190:37269)
0810 19:33:11.409921 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:11.409927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:11.410332 (+ 405us) server_negotiation.cc:407] Connection header received
0810 19:33:11.410644 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:11.410650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:11.410743 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:11.410857 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:11.412011 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.413365 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.414681 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.415110 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.415255 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:11.419613 (+ 4358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:43675: BlockingRecv error: recv got EOF from 127.29.233.131:43675 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 19:33:11 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:11 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:11.438422 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:11.427345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:58317)
0810 19:33:11.428140 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:11.428162 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:11.428313 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:11.428638 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:11.428645 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:11.429183 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:11.429402 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:11.429416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.430544 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.430552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:11.432245 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:11.432266 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.433013 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.433022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:11.433233 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:11.433916 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:11.433945 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:11.438001 (+ 4056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":525,"threads_started":1}
W20260810 19:33:11.438431 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:11.427437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:58317 (local address 127.29.233.190:37269)
0810 19:33:11.427724 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:11.427731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:11.428257 (+ 526us) server_negotiation.cc:407] Connection header received
0810 19:33:11.428427 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:11.428431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:11.428491 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:11.428576 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:11.429551 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.430405 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.432414 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.432838 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.432939 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:11.438151 (+ 5212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:58317: BlockingRecv error: recv got EOF from 127.29.233.129:58317 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 19:33:11 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:11 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:11.947697 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:11.937887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:55277)
0810 19:33:11.938475 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:11.938492 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:11.938655 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:11.939074 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:11.939078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:11.939509 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:11.939663 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:11.939672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.940760 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.940764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:11.941918 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:11.941933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.942805 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.942811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:11.942958 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:11.943703 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:11.943722 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:11.947310 (+ 3588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":322,"threads_started":1}
W20260810 19:33:11.947737 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:11.938035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:55277 (local address 127.29.233.190:37269)
0810 19:33:11.938333 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:11.938340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:11.938585 (+ 245us) server_negotiation.cc:407] Connection header received
0810 19:33:11.938786 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:11.938794 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:11.938915 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:11.939031 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:11.939852 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.940593 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.942080 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:11.942637 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:11.942737 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:11.947518 (+ 4781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:55277: BlockingRecv error: recv got EOF from 127.29.233.130:55277 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 10 19:33:12 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:12 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:12.430862 8392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:12.420899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:40531)
0810 19:33:12.421475 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:12.421493 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:12.421611 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:12.421973 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:12.421976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:12.422408 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:12.422558 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:12.422567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.423664 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.423668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:12.424613 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:12.424625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.425437 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.425441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:12.425555 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:12.426324 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:12.426395 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:12.430458 (+ 4063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":323,"threads_started":1}
W20260810 19:33:12.435937 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:12.421055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:40531 (local address 127.29.233.190:37269)
0810 19:33:12.421276 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:12.421281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:12.421587 (+ 306us) server_negotiation.cc:407] Connection header received
0810 19:33:12.421771 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:12.421775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:12.421840 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:12.421932 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:12.422700 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.423518 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.424784 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.425247 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.425362 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:12.435656 (+ 10294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:40531: BlockingRecv error: recv got EOF from 127.29.233.131:40531 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 10 19:33:12 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:12 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 19:33:12.452028 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:12.439563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:52199 (local address 127.29.233.190:37269)
0810 19:33:12.439781 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:12.439790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:12.440212 (+ 422us) server_negotiation.cc:407] Connection header received
0810 19:33:12.440419 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:12.440426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:12.440537 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:12.440650 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:12.441691 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.442752 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.444626 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.445358 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.445466 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:12.451674 (+ 6208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:52199: BlockingRecv error: recv got EOF from 127.29.233.129:52199 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 19:33:12.452615 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:12.439518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:52199)
0810 19:33:12.440096 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:12.440119 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:12.440275 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:12.440716 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:12.440722 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:12.441279 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:12.441514 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:12.441530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.442923 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.442928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:12.444444 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:12.444463 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.445736 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.445744 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:12.445982 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:12.446780 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:12.446808 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:12.452280 (+ 5472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":307,"threads_started":1}
Aug 10 19:33:12 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:12 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:12.962810 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:12.948714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:52859)
0810 19:33:12.949305 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:12.949330 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:12.949495 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:12.949798 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:12.949802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:12.950223 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:12.950519 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:12.950529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.951592 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.951596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:12.952537 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:12.952548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.953305 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.953310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:12.953454 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:12.959249 (+ 5795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:12.959271 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:12.962555 (+ 3284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":345,"threads_started":1}
W20260810 19:33:12.963053 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:12.948826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:52859 (local address 127.29.233.190:37269)
0810 19:33:12.949116 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:12.949125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:12.949438 (+ 313us) server_negotiation.cc:407] Connection header received
0810 19:33:12.949603 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:12.949607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:12.949670 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:12.949751 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:12.950663 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.951452 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.952684 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:12.953147 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:12.953243 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:12.962793 (+ 9550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:52859: BlockingRecv error: recv got EOF from 127.29.233.130:52859 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 10 19:33:13 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:13 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:13.441054 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:13.432337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:33453)
0810 19:33:13.432891 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:13.432907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:13.433031 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:13.433381 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:13.433385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:13.433810 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:13.434005 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:13.434013 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.435131 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.435135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:13.436123 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:13.436136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.436892 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.436897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:13.437037 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:13.437751 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:13.437769 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:13.440796 (+ 3027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":297,"threads_started":1}
W20260810 19:33:13.441205 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:13.432409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:33453 (local address 127.29.233.190:37269)
0810 19:33:13.432667 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:13.432672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:13.432990 (+ 318us) server_negotiation.cc:407] Connection header received
0810 19:33:13.433145 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:13.433149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:13.433216 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:13.433309 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:13.434147 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.434989 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.436264 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.436706 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.436796 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:13.440975 (+ 4179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:33453: BlockingRecv error: recv got EOF from 127.29.233.131:33453 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 10 19:33:13 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:13 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:13.462303 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:13.453680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:36809)
0810 19:33:13.454234 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:13.454250 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:13.454382 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:13.454699 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:13.454702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:13.455110 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:13.455263 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:13.455272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.456283 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.456287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:13.457229 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:13.457241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.458002 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.458025 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:13.458155 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:13.458863 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:13.458881 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:13.462010 (+ 3129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":338,"threads_started":1}
W20260810 19:33:13.462653 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:13.453707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:36809 (local address 127.29.233.190:37269)
0810 19:33:13.453883 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:13.453887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:13.454372 (+ 485us) server_negotiation.cc:407] Connection header received
0810 19:33:13.454483 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:13.454487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:13.454566 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:13.454662 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:13.455381 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.456129 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.457369 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.457858 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.457949 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:13.462288 (+ 4339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:36809: BlockingRecv error: recv got EOF from 127.29.233.129:36809 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 19:33:13 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:13 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:13.972945 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:13.963884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:42877)
0810 19:33:13.964443 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:13.964465 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:13.964599 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:13.965054 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:13.965062 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:13.965630 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:13.965810 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:13.965819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.966997 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.967001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:13.967979 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:13.967991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.968719 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.968723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:13.968870 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:13.969488 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:13.969508 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:13.972716 (+ 3208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":319,"threads_started":1}
W20260810 19:33:13.973294 8398 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:13.963931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:42877 (local address 127.29.233.190:37269)
0810 19:33:13.964695 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:13.964703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:13.964726 (+ 23us) server_negotiation.cc:407] Connection header received
0810 19:33:13.964785 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:13.964790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:13.964875 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:13.964979 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:13.965957 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.966847 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.968113 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:13.968579 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:13.968675 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:13.973003 (+ 4328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:42877: BlockingRecv error: recv got EOF from 127.29.233.130:42877 (error 108)
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":493,"threads_started":1}
Aug 10 19:33:14 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:14 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:14.451531 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:14.442243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:51821)
0810 19:33:14.443041 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:14.443059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:14.443249 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:14.443723 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:14.443729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:14.444361 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:14.444557 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:14.444567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.445595 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.445599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:14.446686 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:14.446699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.447440 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.447444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:14.447580 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:14.448211 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:14.448229 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:14.451264 (+ 3035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":472,"threads_started":1}
W20260810 19:33:14.451730 8398 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:14.442506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:51821 (local address 127.29.233.190:37269)
0810 19:33:14.442846 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:14.442853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:14.443182 (+ 329us) server_negotiation.cc:407] Connection header received
0810 19:33:14.443461 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:14.443466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:14.443557 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:14.443654 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:14.444696 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.445462 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.446879 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.447297 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.447413 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:14.451480 (+ 4067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:51821: BlockingRecv error: recv got EOF from 127.29.233.131:51821 (error 108)
Metrics: {"server-negotiator.queue_time_us":192}
Aug 10 19:33:14 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:14 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:14.472568 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:14.463452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:55931)
0810 19:33:14.464080 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:14.464097 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:14.464213 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:14.464546 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:14.464550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:14.464948 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:14.465119 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:14.465128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.466213 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.466217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:14.467507 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:14.467520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.468567 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.468572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:14.468690 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:14.469290 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:14.469307 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:14.472293 (+ 2986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":338,"threads_started":1}
W20260810 19:33:14.472855 8398 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:14.463519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:55931 (local address 127.29.233.190:37269)
0810 19:33:14.463692 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:14.463697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:14.464177 (+ 480us) server_negotiation.cc:407] Connection header received
0810 19:33:14.464313 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:14.464317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:14.464416 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:14.464508 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:14.465248 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.466081 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.467661 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.468368 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.468494 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:14.472556 (+ 4062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:55931: BlockingRecv error: recv got EOF from 127.29.233.129:55931 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 19:33:14 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:14 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:14.982801 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:14.974045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:60123)
0810 19:33:14.974766 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:14.974792 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:14.974954 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:14.975405 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:14.975411 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:14.975896 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:14.976069 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:14.976079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.977133 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.977137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:14.978073 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:14.978085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.978875 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.978880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:14.979011 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:14.979698 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:14.979715 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:14.982561 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":373,"threads_started":1}
W20260810 19:33:14.983017 8403 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:14.974225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:60123 (local address 127.29.233.190:37269)
0810 19:33:14.975039 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:14.975046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:14.975068 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:33:14.975122 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:14.975127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:14.975223 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:14.975345 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:14.976210 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.976975 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.978208 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:14.978707 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:14.978803 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:14.982768 (+ 3965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:60123: BlockingRecv error: recv got EOF from 127.29.233.130:60123 (error 108)
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":519,"threads_started":1}
Aug 10 19:33:15 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:15 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:15.461665 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:15.452592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:49603)
0810 19:33:15.453077 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:15.453095 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:15.453218 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:15.453527 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:15.453531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:15.453953 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:15.454112 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:15.454120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.455221 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.455226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:15.456196 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:15.456209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.456974 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.456978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:15.457117 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:15.457840 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:15.457857 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:15.461384 (+ 3527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
W20260810 19:33:15.461920 8403 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:15.452700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:49603 (local address 127.29.233.190:37269)
0810 19:33:15.452946 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:15.452953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:15.453192 (+ 239us) server_negotiation.cc:407] Connection header received
0810 19:33:15.453327 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:15.453331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:15.453403 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:15.453487 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:15.454269 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.455063 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.456345 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.456788 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.456908 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:15.461641 (+ 4733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:49603: BlockingRecv error: recv got EOF from 127.29.233.131:49603 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 10 19:33:15 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:15 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:15.482259 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:15.473763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:53869)
0810 19:33:15.474234 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:15.474251 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:15.474405 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:15.474714 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:15.474717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:15.475085 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:15.475264 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:15.475272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.476365 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.476369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:15.477494 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:15.477508 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.478257 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.478262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:15.478417 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:15.479076 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:15.479093 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:15.482001 (+ 2908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
W20260810 19:33:15.482597 8403 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:15.473822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53869 (local address 127.29.233.190:37269)
0810 19:33:15.474144 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:15.474150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:15.474402 (+ 252us) server_negotiation.cc:407] Connection header received
0810 19:33:15.474508 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:15.474512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:15.474594 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:15.474672 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:15.475446 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.476236 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.477653 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.478082 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.478200 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:15.482235 (+ 4035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:53869: BlockingRecv error: recv got EOF from 127.29.233.129:53869 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Aug 10 19:33:15 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:15 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:15.992731 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:15.983886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:47853)
0810 19:33:15.984499 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:15.984524 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:15.984653 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:15.984964 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:15.984967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:15.985357 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:15.985540 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:15.985550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.986710 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.986714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:15.987650 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:15.987662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.988425 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.988430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:15.988563 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:15.989302 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:15.989322 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:15.992488 (+ 3166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":248,"threads_started":1}
W20260810 19:33:15.993166 8408 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:15.984000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:47853 (local address 127.29.233.190:37269)
0810 19:33:15.984654 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:15.984658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:15.984674 (+ 16us) server_negotiation.cc:407] Connection header received
0810 19:33:15.984740 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:15.984744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:15.984814 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:15.984901 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:15.985694 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.986554 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.987797 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:15.988242 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:15.988339 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:15.992896 (+ 4557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:47853: BlockingRecv error: recv got EOF from 127.29.233.130:47853 (error 108)
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":424,"threads_started":1}
Aug 10 19:33:16 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:16 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.131@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:16.471613 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:16.462812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.131:50473)
0810 19:33:16.463339 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:16.463356 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:16.463527 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:16.463902 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:16.463907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:16.464304 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:16.464496 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:16.464504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.465623 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.465629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:16.466589 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:16.466600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.467329 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.467335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:16.467519 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:16.468184 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:16.468206 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:16.471370 (+ 3164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":296,"threads_started":1}
W20260810 19:33:16.471906 8408 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:16.462893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:50473 (local address 127.29.233.190:37269)
0810 19:33:16.463114 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:16.463123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:16.463465 (+ 342us) server_negotiation.cc:407] Connection header received
0810 19:33:16.463680 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:16.463690 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:16.463757 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:16.463867 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:16.464658 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.465449 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.466747 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.467166 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.467303 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:16.471641 (+ 4338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.131:50473: BlockingRecv error: recv got EOF from 127.29.233.131:50473 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 10 19:33:16 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:16 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.129@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:16.493300 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:16.483421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.129:53889)
0810 19:33:16.483937 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:16.483955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:16.484074 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:16.484408 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:16.484412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:16.484833 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:16.485010 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:16.485019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.486039 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.486043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:16.487033 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:16.487088 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.488426 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.488433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:16.488678 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:16.489569 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:16.489596 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:16.492940 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":280,"threads_started":1}
W20260810 19:33:16.493937 8408 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:16.483589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53889 (local address 127.29.233.190:37269)
0810 19:33:16.483830 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:16.483837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:16.484035 (+ 198us) server_negotiation.cc:407] Connection header received
0810 19:33:16.484168 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:16.484172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:16.484245 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:16.484327 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:16.485141 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.485900 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.487307 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.488191 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.488355 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:16.493300 (+ 4945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.129:53889: BlockingRecv error: recv got EOF from 127.29.233.129:53889 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 10 19:33:16 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 19:33:17 dist-test-slave-tm1g krb5kdc[7267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.233.130@KRBTEST.COM for oryx/127.29.233.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 19:33:17.003418 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:16.994077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:37269 (local address 127.29.233.130:45393)
0810 19:33:16.995037 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:16.995056 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:16.995174 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:16.995534 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:16.995538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:16.995979 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:16.996152 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:16.996161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.997203 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.997208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:16.998175 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:16.998187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.999002 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.999007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:16.999145 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:16.999905 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:16.999929 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:17.003151 (+ 3222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.233.190:37269: Server oryx/127.29.233.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":582,"threads_started":1}
W20260810 19:33:17.003667 8413 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:16.994206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:45393 (local address 127.29.233.190:37269)
0810 19:33:16.995089 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:16.995093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:16.995115 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:33:16.995292 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:16.995296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:16.995361 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:16.995459 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:16.996299 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.997049 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.998333 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:16.998815 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:16.998932 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:17.003387 (+ 4455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.233.130:45393: BlockingRecv error: recv got EOF from 127.29.233.130:45393 (error 108)
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":750,"thread_start_us":566,"threads_started":1}
I20260810 19:33:17.287279 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 7903
I20260810 19:33:17.300889 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:17.288433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.29.233.129:34519)
0810 19:33:17.289025 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:17.289033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:17.289061 (+ 28us) server_negotiation.cc:407] Connection header received
0810 19:33:17.289124 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:17.289130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:17.289361 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:17.289543 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:17.291562 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:17.292674 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:17.294167 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:17.294834 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:17.294943 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:17.297546 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:17.297554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:17.297569 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:17.297600 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:17.300362 (+ 2762us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.233.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 19:33:17.300655 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41288: Request ticket server kudu/127.29.233.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":311,"threads_started":1}
W20260810 19:33:17.301043 8415 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41288: Request ticket server kudu/127.29.233.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 19:33:17.301185 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 7903: Not authorized: Client connection negotiation failed: client connection to 127.29.233.129:34519: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.233.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 19:33:17.301438 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 7903
I20260810 19:33:17.323076 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8068
I20260810 19:33:17.336299 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:17.324491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.29.233.130:45997)
0810 19:33:17.325138 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:17.325146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:17.325177 (+ 31us) server_negotiation.cc:407] Connection header received
0810 19:33:17.325240 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:17.325247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:17.325503 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:17.325664 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:17.327181 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:17.328290 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:17.329573 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:17.330164 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:17.330267 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:17.333099 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:17.333108 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:17.333122 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:17.333166 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:17.335779 (+ 2613us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.233.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 19:33:17.336058 (+ 279us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39856: Request ticket server kudu/127.29.233.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":327,"threads_started":1}
W20260810 19:33:17.336434 8416 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39856: Request ticket server kudu/127.29.233.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 19:33:17.336519 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8068: Not authorized: Client connection negotiation failed: client connection to 127.29.233.130:45997: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.233.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 19:33:17.336680 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 8068
I20260810 19:33:17.357281 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8211
I20260810 19:33:17.370093 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:17.358376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.29.233.131:35445)
0810 19:33:17.359010 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:17.359017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:17.359072 (+ 55us) server_negotiation.cc:407] Connection header received
0810 19:33:17.359151 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:17.359161 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:17.359448 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:17.359606 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:17.360950 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:17.362077 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:17.363510 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:17.364067 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:17.364174 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:17.366937 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:17.366947 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:17.366962 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:17.367013 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:17.369532 (+ 2519us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.233.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 19:33:17.369804 (+ 272us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46772: Request ticket server kudu/127.29.233.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":321,"threads_started":1}
W20260810 19:33:17.370211 8418 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46772: Request ticket server kudu/127.29.233.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 19:33:17.370342 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8211: Not authorized: Client connection negotiation failed: client connection to 127.29.233.131:35445: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.233.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 19:33:17.370546 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 8211
I20260810 19:33:17.392030 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 7828
I20260810 19:33:17.393499 7896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:17.517534 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 7828
2026-08-10T19:33:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21553 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T19:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:33:17Z Disabled control of system clock
I20260810 19:33:17.614184 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:44609
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:44609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 19:33:17.979424 8423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:17.979948 8423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:17.980118 8423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:17.988691 8423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:33:17.988826 8423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:17.988919 8423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:17.989023 8423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:33:17.989120 8423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:33:17.999586 8423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:44609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:44609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:18.001102 8423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:18.003144 8423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:18.017247 8428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:18.017182 8429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:18.017223 8431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:18.017577 8423 server_base.cc:1034] running on GCE node
I20260810 19:33:18.018741 8423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:18.020372 8423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:18.021622 8423 hybrid_clock.cc:648] HybridClock initialized: now 1786390398021565 us; error 40 us; skew 500 ppm
I20260810 19:33:18.022081 8423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:18.029722 8423 webserver.cc:460] Webserver started at http://127.29.233.190:42873/ using document root <none> and password file <none>
I20260810 19:33:18.030697 8423 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:18.030827 8423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:18.031188 8423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:18.034492 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "c276de7a00024ac9b6084c86efc45d0c"
format_stamp: "Formatted at 2026-08-10 19:33:18 on dist-test-slave-tm1g"
server_key: "4fca56c495640b234be12b8afb294414"
server_key_iv: "b1fa562619f8574018baa37d4a93a322"
server_key_version: "encryptionkey@0"
I20260810 19:33:18.035383 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "c276de7a00024ac9b6084c86efc45d0c"
format_stamp: "Formatted at 2026-08-10 19:33:18 on dist-test-slave-tm1g"
server_key: "4fca56c495640b234be12b8afb294414"
server_key_iv: "b1fa562619f8574018baa37d4a93a322"
server_key_version: "encryptionkey@0"
I20260810 19:33:18.041055 8423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:33:18.044921 8437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:18.046620 8423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:33:18.046833 8423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "c276de7a00024ac9b6084c86efc45d0c"
format_stamp: "Formatted at 2026-08-10 19:33:18 on dist-test-slave-tm1g"
server_key: "4fca56c495640b234be12b8afb294414"
server_key_iv: "b1fa562619f8574018baa37d4a93a322"
server_key_version: "encryptionkey@0"
I20260810 19:33:18.047084 8423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:18.067651 8423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:18.068608 8423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:18.068924 8423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:18.089846 8423 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:44609
I20260810 19:33:18.089865 8496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:44609 every 8 connection(s)
I20260810 19:33:18.091368 8423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:33:18.094002 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 8423
I20260810 19:33:18.094209 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:33:18.095165 30630 external_mini_cluster.cc:1428] Setting key 65e07ceebf4e210961cb01a0d1036e3e
I20260810 19:33:18.096024 8497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:33:18.110451 8497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c: Bootstrap starting.
I20260810 19:33:18.111232 8501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:18.097426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.29.233.190:44609)
0810 19:33:18.098518 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:18.098528 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:18.098570 (+ 42us) server_negotiation.cc:407] Connection header received
0810 19:33:18.101025 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:18.101050 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:18.101549 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:18.101917 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:18.102718 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:18.104079 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:18.105985 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:18.106909 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:18.107134 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:18.108421 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:18.108432 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:18.108447 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:18.109041 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:18.109127 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:18.109149 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:18.109255 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:18.109554 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 19:33:18.109962 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":397,"threads_started":1}
I20260810 19:33:18.113814 8497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c: Neither blocks nor log segments found. Creating new log.
I20260810 19:33:18.114955 8497 log.cc:824] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c: Log is configured to *not* fsync() on all Append() calls
I20260810 19:33:18.117803 8497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c: No bootstrap required, opened a new log
I20260810 19:33:18.125267 8497 raft_consensus.cc:348] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c276de7a00024ac9b6084c86efc45d0c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44609 } }
I20260810 19:33:18.125710 8497 raft_consensus.cc:374] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:33:18.125834 8497 raft_consensus.cc:729] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c276de7a00024ac9b6084c86efc45d0c, State: Initialized, Role: FOLLOWER
I20260810 19:33:18.126545 8497 consensus_queue.cc:260] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c276de7a00024ac9b6084c86efc45d0c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44609 } }
I20260810 19:33:18.126763 8497 raft_consensus.cc:388] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:33:18.126863 8497 raft_consensus.cc:482] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:33:18.127000 8497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:33:18.129153 8497 raft_consensus.cc:504] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c276de7a00024ac9b6084c86efc45d0c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44609 } }
I20260810 19:33:18.129630 8497 leader_election.cc:302] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c276de7a00024ac9b6084c86efc45d0c; no voters:
I20260810 19:33:18.130239 8497 leader_election.cc:288] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:33:18.130499 8503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:33:18.131234 8503 raft_consensus.cc:686] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [term 1 LEADER]: Becoming Leader. State: Replica: c276de7a00024ac9b6084c86efc45d0c, State: Running, Role: LEADER
I20260810 19:33:18.131860 8503 consensus_queue.cc:237] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c276de7a00024ac9b6084c86efc45d0c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44609 } }
I20260810 19:33:18.132576 8497 sys_catalog.cc:564] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:33:18.134822 8505 sys_catalog.cc:455] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c276de7a00024ac9b6084c86efc45d0c. Latest consensus state: current_term: 1 leader_uuid: "c276de7a00024ac9b6084c86efc45d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c276de7a00024ac9b6084c86efc45d0c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44609 } } }
I20260810 19:33:18.135214 8505 sys_catalog.cc:458] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:18.135704 8504 sys_catalog.cc:455] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c276de7a00024ac9b6084c86efc45d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c276de7a00024ac9b6084c86efc45d0c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44609 } } }
I20260810 19:33:18.136046 8504 sys_catalog.cc:458] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:18.137794 8514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:33:18.146072 8514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:33:18.156383 8514 catalog_manager.cc:1347] Generated new cluster ID: 8a420b375d024838a76bd99b02cf7f48
I20260810 19:33:18.156556 8514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:33:18.172999 8514 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:33:18.174458 8514 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:33:18.183125 8514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c276de7a00024ac9b6084c86efc45d0c: Generated new TSK 0
I20260810 19:33:18.184302 8514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:33:18.200682 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44609
--builtin_ntp_servers=127.29.233.148:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 19:33:18.569357 8526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:18.569658 8526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:18.569801 8526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:18.577479 8526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:18.577590 8526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:18.577772 8526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:33:18.587611 8526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:18.589057 8526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:18.591043 8526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:18.604724 8532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:18.604811 8536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:18.604756 8533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:18.605684 8526 server_base.cc:1034] running on GCE node
I20260810 19:33:18.606274 8526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:18.607278 8526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:18.608529 8526 hybrid_clock.cc:648] HybridClock initialized: now 1786390398608491 us; error 55 us; skew 500 ppm
I20260810 19:33:18.608862 8526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:18.611611 8526 webserver.cc:460] Webserver started at http://127.29.233.129:45721/ using document root <none> and password file <none>
I20260810 19:33:18.612432 8526 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:18.612581 8526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:18.612929 8526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:18.615962 8526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "98af2b79652441e49285a050540776da"
format_stamp: "Formatted at 2026-08-10 19:33:18 on dist-test-slave-tm1g"
server_key: "eb0665be0c7ad27589d67e578b7ba1eb"
server_key_iv: "fbb7e74820c48696d347426eb89fee62"
server_key_version: "encryptionkey@0"
I20260810 19:33:18.616796 8526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "98af2b79652441e49285a050540776da"
format_stamp: "Formatted at 2026-08-10 19:33:18 on dist-test-slave-tm1g"
server_key: "eb0665be0c7ad27589d67e578b7ba1eb"
server_key_iv: "fbb7e74820c48696d347426eb89fee62"
server_key_version: "encryptionkey@0"
I20260810 19:33:18.622471 8526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:33:18.626088 8542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:18.627547 8526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:33:18.627743 8526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "98af2b79652441e49285a050540776da"
format_stamp: "Formatted at 2026-08-10 19:33:18 on dist-test-slave-tm1g"
server_key: "eb0665be0c7ad27589d67e578b7ba1eb"
server_key_iv: "fbb7e74820c48696d347426eb89fee62"
server_key_version: "encryptionkey@0"
I20260810 19:33:18.627974 8526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:18.653739 8526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:18.654771 8526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:18.655113 8526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:18.656131 8526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:18.657655 8526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:18.657785 8526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:18.657922 8526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:18.658012 8526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:18.692898 8526 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:41169
I20260810 19:33:18.692993 8663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:41169 every 8 connection(s)
I20260810 19:33:18.694296 8526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:33:18.704032 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 8526
I20260810 19:33:18.704257 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:33:18.704996 30630 external_mini_cluster.cc:1428] Setting key c12c4f942650f85fa3fc547da1518bc1
I20260810 19:33:18.707901 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:18.697120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:45467 (local address 127.29.233.190:44609)
0810 19:33:18.697634 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:18.697639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:18.698814 (+ 1175us) server_negotiation.cc:407] Connection header received
0810 19:33:18.700753 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:18.700759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:18.700856 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:18.700985 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:18.702070 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:18.702882 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:18.704629 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:18.705038 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:18.705189 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:18.706902 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:18.706907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:18.706910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:18.706928 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:18.706971 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:18.706984 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:18.707120 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:18.707423 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 19:33:18.707547 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":288,"threads_started":1}
I20260810 19:33:18.708488 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44609
--builtin_ntp_servers=127.29.233.148:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 19:33:18.709587 8668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:18.697661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:44609 (local address 127.29.233.129:45467)
0810 19:33:18.698641 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:18.698686 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:18.700464 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:18.701213 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:18.701222 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:18.701312 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:18.701883 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:18.701900 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:18.703061 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:18.703066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:18.704455 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:18.704468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:18.705280 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:18.705307 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:18.705561 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:18.706507 (+ 946us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:33:18.706510 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:18.706585 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:33:18.706697 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:18.707137 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:18.707150 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:18.707163 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 19:33:18.707384 (+ 221us) client_negotiation.cc:240] Negotiation successful
0810 19:33:18.707739 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":289,"threads_started":1}
I20260810 19:33:18.715569 8664 heartbeater.cc:347] Connected to a master server at 127.29.233.190:44609
I20260810 19:33:18.715950 8664 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:18.716912 8664 heartbeater.cc:511] Master 127.29.233.190:44609 requested a full tablet report, sending...
I20260810 19:33:18.719597 8456 ts_manager.cc:194] Registered new tserver with Master: 98af2b79652441e49285a050540776da (127.29.233.129:41169)
I20260810 19:33:18.721496 8456 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.129:45467
W20260810 19:33:19.068331 8670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:19.068650 8670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:19.068819 8670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:19.076337 8670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:19.076442 8670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:19.076620 8670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:33:19.086253 8670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:19.087683 8670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:19.089676 8670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:19.104568 8676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:19.104952 8677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:19.105247 8679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:19.105698 8670 server_base.cc:1034] running on GCE node
I20260810 19:33:19.106377 8670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:19.107372 8670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:19.108629 8670 hybrid_clock.cc:648] HybridClock initialized: now 1786390399108579 us; error 42 us; skew 500 ppm
I20260810 19:33:19.108964 8670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:19.111711 8670 webserver.cc:460] Webserver started at http://127.29.233.130:33851/ using document root <none> and password file <none>
I20260810 19:33:19.112509 8670 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:19.112630 8670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:19.112960 8670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:19.116051 8670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "50ad64a64c86460fb3637edfb93751d6"
format_stamp: "Formatted at 2026-08-10 19:33:19 on dist-test-slave-tm1g"
server_key: "33e5f5d7e71588f152a8b5e1168f2170"
server_key_iv: "e6446f74341a9fb63ecfa2a9d3073fc9"
server_key_version: "encryptionkey@0"
I20260810 19:33:19.116879 8670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "50ad64a64c86460fb3637edfb93751d6"
format_stamp: "Formatted at 2026-08-10 19:33:19 on dist-test-slave-tm1g"
server_key: "33e5f5d7e71588f152a8b5e1168f2170"
server_key_iv: "e6446f74341a9fb63ecfa2a9d3073fc9"
server_key_version: "encryptionkey@0"
I20260810 19:33:19.122108 8670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 19:33:19.125820 8685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:19.127204 8670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:33:19.127384 8670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "50ad64a64c86460fb3637edfb93751d6"
format_stamp: "Formatted at 2026-08-10 19:33:19 on dist-test-slave-tm1g"
server_key: "33e5f5d7e71588f152a8b5e1168f2170"
server_key_iv: "e6446f74341a9fb63ecfa2a9d3073fc9"
server_key_version: "encryptionkey@0"
I20260810 19:33:19.127631 8670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:19.159605 8670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:19.160561 8670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:19.160894 8670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:19.161943 8670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:19.163601 8670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:19.163691 8670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:19.163832 8670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:19.163913 8670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:19.198822 8670 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:42181
I20260810 19:33:19.198925 8811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:42181 every 8 connection(s)
I20260810 19:33:19.200197 8670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:33:19.204094 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 8670
I20260810 19:33:19.204283 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:33:19.204839 30630 external_mini_cluster.cc:1428] Setting key 19cfdffdcd3fa2db78829fcb3ca50b5a
I20260810 19:33:19.207731 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44609
--builtin_ntp_servers=127.29.233.148:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 19:33:19.213279 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:19.203018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:52037 (local address 127.29.233.190:44609)
0810 19:33:19.203206 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:19.203210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:19.204788 (+ 1578us) server_negotiation.cc:407] Connection header received
0810 19:33:19.206843 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:19.206849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:19.206936 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:19.207047 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:19.208157 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.208889 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.210495 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.210895 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.211040 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:19.212448 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:19.212453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:19.212457 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:19.212475 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:19.212506 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:19.212518 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:19.212616 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:19.212834 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 19:33:19.212939 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:33:19.214349 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:19.203555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:44609 (local address 127.29.233.130:52037)
0810 19:33:19.204613 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:19.204657 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:19.206538 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:19.207274 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:19.207282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:19.207369 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:19.207976 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:19.208001 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.209050 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.209055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:19.210282 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:19.210293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.211125 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.211132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:19.211375 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:19.212126 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:33:19.212129 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:19.212167 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:33:19.212241 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:19.212614 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:19.212627 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:19.212637 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 19:33:19.212834 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 19:33:19.213232 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":369,"threads_started":1}
I20260810 19:33:19.215898 8812 heartbeater.cc:347] Connected to a master server at 127.29.233.190:44609
I20260810 19:33:19.216209 8812 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:19.217005 8812 heartbeater.cc:511] Master 127.29.233.190:44609 requested a full tablet report, sending...
I20260810 19:33:19.218554 8456 ts_manager.cc:194] Registered new tserver with Master: 50ad64a64c86460fb3637edfb93751d6 (127.29.233.130:42181)
I20260810 19:33:19.219376 8456 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.130:52037
W20260810 19:33:19.560657 8816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:19.560961 8816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:19.561106 8816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:19.568712 8816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:19.568818 8816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:19.569010 8816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:33:19.578548 8816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:19.579924 8816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:19.581782 8816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:19.596050 8826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:19.596091 8822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:19.596021 8823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:19.596856 8816 server_base.cc:1034] running on GCE node
I20260810 19:33:19.597581 8816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:19.598701 8816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:19.599954 8816 hybrid_clock.cc:648] HybridClock initialized: now 1786390399599891 us; error 62 us; skew 500 ppm
I20260810 19:33:19.600324 8816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:19.602825 8816 webserver.cc:460] Webserver started at http://127.29.233.131:41457/ using document root <none> and password file <none>
I20260810 19:33:19.603564 8816 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:19.603683 8816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:19.604051 8816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:19.607044 8816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "b54485ff6660445abbb6251f2fa62faa"
format_stamp: "Formatted at 2026-08-10 19:33:19 on dist-test-slave-tm1g"
server_key: "d389655d5e50284facc1f638956487de"
server_key_iv: "7f3f2e0f4b3a64a31d772eb6e5c8f200"
server_key_version: "encryptionkey@0"
I20260810 19:33:19.607887 8816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "b54485ff6660445abbb6251f2fa62faa"
format_stamp: "Formatted at 2026-08-10 19:33:19 on dist-test-slave-tm1g"
server_key: "d389655d5e50284facc1f638956487de"
server_key_iv: "7f3f2e0f4b3a64a31d772eb6e5c8f200"
server_key_version: "encryptionkey@0"
I20260810 19:33:19.613490 8816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:33:19.617178 8833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:19.618734 8816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:33:19.618924 8816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "b54485ff6660445abbb6251f2fa62faa"
format_stamp: "Formatted at 2026-08-10 19:33:19 on dist-test-slave-tm1g"
server_key: "d389655d5e50284facc1f638956487de"
server_key_iv: "7f3f2e0f4b3a64a31d772eb6e5c8f200"
server_key_version: "encryptionkey@0"
I20260810 19:33:19.619151 8816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:19.639601 8816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:19.640515 8816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:19.640856 8816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:19.642016 8816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:19.643626 8816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:19.643723 8816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:19.643870 8816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:19.644001 8816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:19.679185 8816 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:37371
I20260810 19:33:19.679219 8957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:37371 every 8 connection(s)
I20260810 19:33:19.680707 8816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:33:19.689395 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 8816
I20260810 19:33:19.689570 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:33:19.690053 30630 external_mini_cluster.cc:1428] Setting key f9a34f77747a026586ebdc12bf4eadf4
I20260810 19:33:19.694734 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:19.683950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:39319 (local address 127.29.233.190:44609)
0810 19:33:19.684146 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:19.684151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:19.685601 (+ 1450us) server_negotiation.cc:407] Connection header received
0810 19:33:19.687496 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:19.687503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:19.687577 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:19.687674 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:19.688705 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.689560 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.691572 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.692251 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.692388 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:19.693930 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:19.693934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:19.693938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:19.693955 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:19.693990 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:19.694002 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:19.694076 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:19.694382 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 19:33:19.694494 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 19:33:19.696079 8960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:19.684420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:44609 (local address 127.29.233.131:39319)
0810 19:33:19.685435 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:19.685476 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:19.687181 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:19.687880 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:19.687888 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:19.687985 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:19.688542 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:19.688557 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.689753 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.689757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:19.691346 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:19.691387 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.692470 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.692478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:19.692768 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:19.693546 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:33:19.693552 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:19.693607 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:33:19.693728 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:19.694121 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:19.694131 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:19.694140 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 19:33:19.694339 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 19:33:19.694728 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":311,"threads_started":1}
I20260810 19:33:19.697644 8958 heartbeater.cc:347] Connected to a master server at 127.29.233.190:44609
I20260810 19:33:19.697986 8958 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:19.698931 8958 heartbeater.cc:511] Master 127.29.233.190:44609 requested a full tablet report, sending...
I20260810 19:33:19.700397 8456 ts_manager.cc:194] Registered new tserver with Master: b54485ff6660445abbb6251f2fa62faa (127.29.233.131:37371)
I20260810 19:33:19.701233 8456 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.131:39319
I20260810 19:33:19.705976 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 19:33:19.712625 8667 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:19.710791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.29.233.190:44609)
0810 19:33:19.710959 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:19.710964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:19.711461 (+ 497us) server_negotiation.cc:407] Connection header received
0810 19:33:19.711669 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:19.711675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:19.711746 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:19.711857 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:19.712359 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35024: BlockingRecv error: recv got EOF from 127.0.0.1:35024 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:33:19.713624 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8526
I20260810 19:33:19.721815 8967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:19.714718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.29.233.129:41169)
0810 19:33:19.715308 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:19.715315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:19.715339 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:33:19.715396 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:19.715402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:19.715638 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:19.715763 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:19.716166 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.717211 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.718661 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.719236 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.719338 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:19.720430 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:19.720439 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:19.720452 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:19.721059 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:19.721106 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:19.721125 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:19.721215 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:19.721378 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 19:33:19.721507 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":323,"threads_started":1}
I20260810 19:33:19.723206 8659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:19.725621 8664 heartbeater.cc:503] Master 127.29.233.190:44609 was elected leader, sending a full tablet report...
I20260810 19:33:19.812310 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 8526
W20260810 19:33:19.833334 8438 connection.cc:535] server connection from 127.29.233.129:45467 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 19:33:19.833673 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8670
I20260810 19:33:19.841789 8969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:19.834684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.29.233.130:42181)
0810 19:33:19.835275 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:19.835283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:19.835310 (+ 27us) server_negotiation.cc:407] Connection header received
0810 19:33:19.835373 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:19.835381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:19.835613 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:19.835746 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:19.836174 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.837289 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.838702 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.839256 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.839355 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:19.840437 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:19.840446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:19.840461 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:19.841081 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:19.841128 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:19.841148 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:19.841253 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:19.841367 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 19:33:19.841483 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":294,"threads_started":1}
I20260810 19:33:19.843163 8806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:19.926739 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 8670
I20260810 19:33:19.947194 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8816
I20260810 19:33:19.956043 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:19.948419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57422 (local address 127.29.233.131:37371)
0810 19:33:19.949110 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:19.949126 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:19.949158 (+ 32us) server_negotiation.cc:407] Connection header received
0810 19:33:19.949219 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:19.949225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:19.949472 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:19.949606 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:19.950073 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.951263 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.952629 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:19.953203 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:19.953351 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:19.954594 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:19.954607 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:19.954628 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:19.955238 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:19.955285 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:19.955305 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:19.955394 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:19.955607 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 19:33:19.955720 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":385,"threads_started":1}
I20260810 19:33:19.957335 8953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:20.041847 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 8816
I20260810 19:33:20.061713 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8423
I20260810 19:33:20.063086 8491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:20.169234 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 8423
2026-08-10T19:33:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2646 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T19:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:33:20Z Disabled control of system clock
I20260810 19:33:20.260226 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:41939
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:41939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 19:33:20.617343 8977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:20.617656 8977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:20.617748 8977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:20.625111 8977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:33:20.625231 8977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:20.625269 8977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:20.625357 8977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:33:20.625406 8977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:33:20.634910 8977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:41939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:41939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:20.636273 8977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:20.638126 8977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:20.650817 8984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:20.650887 8985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:20.650918 8988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:20.651741 8977 server_base.cc:1034] running on GCE node
I20260810 19:33:20.652495 8977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:20.653867 8977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:20.655079 8977 hybrid_clock.cc:648] HybridClock initialized: now 1786390400655044 us; error 30 us; skew 500 ppm
I20260810 19:33:20.655463 8977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:20.657727 8977 webserver.cc:460] Webserver started at http://127.29.233.190:35421/ using document root <none> and password file <none>
I20260810 19:33:20.658550 8977 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:20.658679 8977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:20.659030 8977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:20.662083 8977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "6d3f5be574f94cac8f0b770d28e6fa84"
format_stamp: "Formatted at 2026-08-10 19:33:20 on dist-test-slave-tm1g"
server_key: "772a77a899f21581d8c9067beb02276d"
server_key_iv: "de73b5c5d50c1d27f588b73512a3a887"
server_key_version: "encryptionkey@0"
I20260810 19:33:20.663049 8977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "6d3f5be574f94cac8f0b770d28e6fa84"
format_stamp: "Formatted at 2026-08-10 19:33:20 on dist-test-slave-tm1g"
server_key: "772a77a899f21581d8c9067beb02276d"
server_key_iv: "de73b5c5d50c1d27f588b73512a3a887"
server_key_version: "encryptionkey@0"
I20260810 19:33:20.668569 8977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 19:33:20.672127 8994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:20.673539 8977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:33:20.673736 8977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "6d3f5be574f94cac8f0b770d28e6fa84"
format_stamp: "Formatted at 2026-08-10 19:33:20 on dist-test-slave-tm1g"
server_key: "772a77a899f21581d8c9067beb02276d"
server_key_iv: "de73b5c5d50c1d27f588b73512a3a887"
server_key_version: "encryptionkey@0"
I20260810 19:33:20.673980 8977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:20.679903 8977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:20.680708 8977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:20.681041 8977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:20.700389 8977 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:41939
I20260810 19:33:20.700429 9052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:41939 every 8 connection(s)
I20260810 19:33:20.701833 8977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:33:20.705834 9053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:33:20.711598 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 8977
I20260810 19:33:20.711838 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:33:20.712661 30630 external_mini_cluster.cc:1428] Setting key 5d005d82b3d83fabf2e32c51c1280d47
I20260810 19:33:20.718168 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84: Bootstrap starting.
I20260810 19:33:20.721702 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:20.715095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.29.233.190:41939)
0810 19:33:20.716258 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:20.716268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:20.716311 (+ 43us) server_negotiation.cc:407] Connection header received
0810 19:33:20.717967 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:20.717989 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:20.718511 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:20.718915 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:20.720049 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:20.720056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:20.720068 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:20.720082 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:20.720144 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:20.720159 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:20.720435 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:20.720587 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 19:33:20.720827 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":375,"threads_started":1}
I20260810 19:33:20.721743 9053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84: Neither blocks nor log segments found. Creating new log.
I20260810 19:33:20.723006 9053 log.cc:824] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84: Log is configured to *not* fsync() on all Append() calls
I20260810 19:33:20.726187 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84: No bootstrap required, opened a new log
I20260810 19:33:20.734268 9053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41939 } }
I20260810 19:33:20.734781 9053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:33:20.734859 9053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3f5be574f94cac8f0b770d28e6fa84, State: Initialized, Role: FOLLOWER
I20260810 19:33:20.735462 9053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41939 } }
I20260810 19:33:20.735682 9053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:33:20.735846 9053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:33:20.736027 9053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:33:20.738677 9053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41939 } }
I20260810 19:33:20.739146 9053 leader_election.cc:302] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d3f5be574f94cac8f0b770d28e6fa84; no voters:
I20260810 19:33:20.739771 9053 leader_election.cc:288] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:33:20.739938 9060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:33:20.740679 9060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [term 1 LEADER]: Becoming Leader. State: Replica: 6d3f5be574f94cac8f0b770d28e6fa84, State: Running, Role: LEADER
I20260810 19:33:20.741281 9060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41939 } }
I20260810 19:33:20.741991 9053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:33:20.744266 9062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d3f5be574f94cac8f0b770d28e6fa84. Latest consensus state: current_term: 1 leader_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41939 } } }
I20260810 19:33:20.744647 9062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:20.746270 9066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:33:20.746225 9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3f5be574f94cac8f0b770d28e6fa84" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41939 } } }
I20260810 19:33:20.746622 9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84 [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:20.751493 9066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:33:20.760646 9066 catalog_manager.cc:1347] Generated new cluster ID: 321dd3a05527433aa20c8dc148ffe399
I20260810 19:33:20.760771 9066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:33:20.789109 9066 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:33:20.789593 9066 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:33:20.805222 9066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d3f5be574f94cac8f0b770d28e6fa84: Generated new TSK 0
I20260810 19:33:20.806458 9066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:33:20.822788 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:41939
--builtin_ntp_servers=127.29.233.148:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 19:33:21.179267 9080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:21.179582 9080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:21.179675 9080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:21.187077 9080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:21.187183 9080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:21.187420 9080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:33:21.196928 9080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:41939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:21.198278 9080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:21.200083 9080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:21.213953 9085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:21.214200 9087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:21.214290 9089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:21.215339 9080 server_base.cc:1034] running on GCE node
I20260810 19:33:21.216022 9080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:21.217041 9080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:21.218290 9080 hybrid_clock.cc:648] HybridClock initialized: now 1786390401218237 us; error 44 us; skew 500 ppm
I20260810 19:33:21.218688 9080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:21.221203 9080 webserver.cc:460] Webserver started at http://127.29.233.129:42669/ using document root <none> and password file <none>
I20260810 19:33:21.221969 9080 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:21.222113 9080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:21.222517 9080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:21.225538 9080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "24fd36bcea194bdb92976e8dc1731593"
format_stamp: "Formatted at 2026-08-10 19:33:21 on dist-test-slave-tm1g"
server_key: "d4050b4315bab44723ff6439b6ef70f1"
server_key_iv: "e441f6a8ea176c18bbba20b6783ca98f"
server_key_version: "encryptionkey@0"
I20260810 19:33:21.226477 9080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "24fd36bcea194bdb92976e8dc1731593"
format_stamp: "Formatted at 2026-08-10 19:33:21 on dist-test-slave-tm1g"
server_key: "d4050b4315bab44723ff6439b6ef70f1"
server_key_iv: "e441f6a8ea176c18bbba20b6783ca98f"
server_key_version: "encryptionkey@0"
I20260810 19:33:21.231863 9080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:33:21.235590 9098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:21.237007 9080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:33:21.237212 9080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "24fd36bcea194bdb92976e8dc1731593"
format_stamp: "Formatted at 2026-08-10 19:33:21 on dist-test-slave-tm1g"
server_key: "d4050b4315bab44723ff6439b6ef70f1"
server_key_iv: "e441f6a8ea176c18bbba20b6783ca98f"
server_key_version: "encryptionkey@0"
I20260810 19:33:21.237480 9080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:21.243870 9080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:21.244732 9080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:21.245043 9080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:21.246194 9080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:21.247983 9080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:21.248073 9080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:21.248152 9080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:21.248207 9080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:21.284581 9080 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39609
I20260810 19:33:21.284639 9223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39609 every 8 connection(s)
I20260810 19:33:21.286022 9080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:33:21.294642 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:21.288868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:49801 (local address 127.29.233.190:41939)
0810 19:33:21.289360 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:21.289368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:21.290699 (+ 1331us) server_negotiation.cc:407] Connection header received
0810 19:33:21.292441 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:21.292446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:21.292505 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:21.292611 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:21.293920 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:21.293924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:21.293927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:21.293945 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:21.293989 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:21.294001 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:21.294099 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:21.294270 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 19:33:21.294418 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":224,"threads_started":1}
I20260810 19:33:21.294876 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 9080
I20260810 19:33:21.295122 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:33:21.295981 30630 external_mini_cluster.cc:1428] Setting key fe2f21693f909e6d09d54e139cc55adb
I20260810 19:33:21.295987 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:21.289448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:41939 (local address 127.29.233.129:49801)
0810 19:33:21.290508 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:21.290571 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:21.292133 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:21.292827 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:21.292836 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:21.292926 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:21.293614 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:33:21.293627 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:21.293665 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:33:21.293752 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:21.294091 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:21.294099 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:21.294104 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 19:33:21.294381 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 19:33:21.294684 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":300,"threads_started":1}
I20260810 19:33:21.297572 9224 heartbeater.cc:347] Connected to a master server at 127.29.233.190:41939
I20260810 19:33:21.297896 9224 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:21.298631 9224 heartbeater.cc:511] Master 127.29.233.190:41939 requested a full tablet report, sending...
I20260810 19:33:21.299252 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:41939
--builtin_ntp_servers=127.29.233.148:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 19:33:21.300748 9014 ts_manager.cc:194] Registered new tserver with Master: 24fd36bcea194bdb92976e8dc1731593 (127.29.233.129:39609)
W20260810 19:33:21.652343 9228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:21.652714 9228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:21.652894 9228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:21.660276 9228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:21.660458 9228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:21.660651 9228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:33:21.670040 9228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:41939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:21.671535 9228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:21.673398 9228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:21.687481 9235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:21.687518 9237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:21.687445 9234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:21.689018 9228 server_base.cc:1034] running on GCE node
I20260810 19:33:21.689718 9228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:21.690804 9228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:21.692044 9228 hybrid_clock.cc:648] HybridClock initialized: now 1786390401692008 us; error 93 us; skew 500 ppm
I20260810 19:33:21.692374 9228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:21.694741 9228 webserver.cc:460] Webserver started at http://127.29.233.130:44295/ using document root <none> and password file <none>
I20260810 19:33:21.695483 9228 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:21.695623 9228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:21.695956 9228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:21.698957 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "c6ca5de76f254645ac8d9ef4fe11ccec"
format_stamp: "Formatted at 2026-08-10 19:33:21 on dist-test-slave-tm1g"
server_key: "ca10a89394f11b421e4b4bcc275e0c4d"
server_key_iv: "bd365e702b10e6b3977ca094eed1a58a"
server_key_version: "encryptionkey@0"
I20260810 19:33:21.699810 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "c6ca5de76f254645ac8d9ef4fe11ccec"
format_stamp: "Formatted at 2026-08-10 19:33:21 on dist-test-slave-tm1g"
server_key: "ca10a89394f11b421e4b4bcc275e0c4d"
server_key_iv: "bd365e702b10e6b3977ca094eed1a58a"
server_key_version: "encryptionkey@0"
I20260810 19:33:21.705235 9228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:33:21.708972 9245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:21.710528 9228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:33:21.710695 9228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "c6ca5de76f254645ac8d9ef4fe11ccec"
format_stamp: "Formatted at 2026-08-10 19:33:21 on dist-test-slave-tm1g"
server_key: "ca10a89394f11b421e4b4bcc275e0c4d"
server_key_iv: "bd365e702b10e6b3977ca094eed1a58a"
server_key_version: "encryptionkey@0"
I20260810 19:33:21.710923 9228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:21.716718 9228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:21.717516 9228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:21.717792 9228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:21.718827 9228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:21.720255 9228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:21.720341 9228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:21.720510 9228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:21.720605 9228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:21.761439 9228 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:43183
I20260810 19:33:21.761538 9367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:43183 every 8 connection(s)
I20260810 19:33:21.762907 9228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:33:21.770987 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 9228
I20260810 19:33:21.771214 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:33:21.771842 30630 external_mini_cluster.cc:1428] Setting key e03a82b9bedb3168346161e60d742667
I20260810 19:33:21.771868 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:21.765766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:46149 (local address 127.29.233.190:41939)
0810 19:33:21.765983 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:21.765987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:21.767450 (+ 1463us) server_negotiation.cc:407] Connection header received
0810 19:33:21.769165 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:21.769170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:21.769231 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:21.769331 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:21.771047 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:21.771052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:21.771056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:21.771076 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:21.771116 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:21.771173 (+ 57us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:21.771401 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:21.771562 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 19:33:21.771624 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 19:33:21.773020 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:21.766246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:41939 (local address 127.29.233.130:46149)
0810 19:33:21.767255 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:21.767292 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:21.768865 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:21.769602 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:21.769610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:21.769699 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:21.770599 (+ 900us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:33:21.770603 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:21.770657 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:33:21.770790 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:21.771322 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:21.771333 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:21.771341 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 19:33:21.771634 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 19:33:21.771907 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":302,"threads_started":1}
I20260810 19:33:21.774576 9368 heartbeater.cc:347] Connected to a master server at 127.29.233.190:41939
I20260810 19:33:21.774916 9368 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:21.775478 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:41939
--builtin_ntp_servers=127.29.233.148:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 19:33:21.775643 9368 heartbeater.cc:511] Master 127.29.233.190:41939 requested a full tablet report, sending...
I20260810 19:33:21.777346 9015 ts_manager.cc:194] Registered new tserver with Master: c6ca5de76f254645ac8d9ef4fe11ccec (127.29.233.130:43183)
W20260810 19:33:22.129541 9374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:22.129869 9374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:22.130041 9374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:22.137542 9374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:22.137689 9374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:22.137864 9374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:33:22.147819 9374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:41939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:22.149253 9374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:22.151314 9374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:22.167510 9380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:22.167704 9379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:22.168318 9374 server_base.cc:1034] running on GCE node
W20260810 19:33:22.168321 9382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:22.169270 9374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:22.170403 9374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:22.171638 9374 hybrid_clock.cc:648] HybridClock initialized: now 1786390402171573 us; error 62 us; skew 500 ppm
I20260810 19:33:22.171985 9374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:33:22.174649 9374 webserver.cc:460] Webserver started at http://127.29.233.131:40787/ using document root <none> and password file <none>
I20260810 19:33:22.175450 9374 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:22.175590 9374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:22.175940 9374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:22.179667 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "1fcea8438a394cba95c5fdea7f6398b1"
format_stamp: "Formatted at 2026-08-10 19:33:22 on dist-test-slave-tm1g"
server_key: "ff656e98e84e10eb933d51c4c09b8d6c"
server_key_iv: "f894cb17f7b09ac65246583a373a18a7"
server_key_version: "encryptionkey@0"
I20260810 19:33:22.180532 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "1fcea8438a394cba95c5fdea7f6398b1"
format_stamp: "Formatted at 2026-08-10 19:33:22 on dist-test-slave-tm1g"
server_key: "ff656e98e84e10eb933d51c4c09b8d6c"
server_key_iv: "f894cb17f7b09ac65246583a373a18a7"
server_key_version: "encryptionkey@0"
I20260810 19:33:22.186221 9374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 19:33:22.190368 9392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:22.191944 9374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 19:33:22.192148 9374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "1fcea8438a394cba95c5fdea7f6398b1"
format_stamp: "Formatted at 2026-08-10 19:33:22 on dist-test-slave-tm1g"
server_key: "ff656e98e84e10eb933d51c4c09b8d6c"
server_key_iv: "f894cb17f7b09ac65246583a373a18a7"
server_key_version: "encryptionkey@0"
I20260810 19:33:22.192394 9374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:22.198534 9374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:22.199393 9374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:22.199698 9374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:22.200834 9374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:22.202525 9374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:22.202641 9374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:22.202776 9374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:22.202868 9374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:22.244586 9374 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:42641
I20260810 19:33:22.244633 9514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:42641 every 8 connection(s)
I20260810 19:33:22.246063 9374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:33:22.248353 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 9374
I20260810 19:33:22.248564 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:33:22.249243 30630 external_mini_cluster.cc:1428] Setting key d54f44b2c2643ac1b9177beeeab1a746
I20260810 19:33:22.261519 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:22.250903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:48229 (local address 127.29.233.190:41939)
0810 19:33:22.251123 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:22.251131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:22.255677 (+ 4546us) server_negotiation.cc:407] Connection header received
0810 19:33:22.258937 (+ 3260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:22.258943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:22.259008 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:22.259102 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:22.260816 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:22.260821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:22.260824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:22.260850 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:22.260886 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:22.260897 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:22.261005 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:22.261235 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 19:33:22.261289 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:33:22.263154 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:22.250505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:41939 (local address 127.29.233.131:48229)
0810 19:33:22.255476 (+ 4971us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:22.255547 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:22.258555 (+ 3008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:22.259443 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:22.259459 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:22.259612 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:22.260426 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:33:22.260431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:22.260491 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:33:22.260621 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:22.261002 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:22.261014 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:22.261026 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 19:33:22.261334 (+ 308us) client_negotiation.cc:240] Negotiation successful
0810 19:33:22.261634 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4195,"thread_start_us":3912,"threads_started":1}
I20260810 19:33:22.264808 9515 heartbeater.cc:347] Connected to a master server at 127.29.233.190:41939
I20260810 19:33:22.265196 9515 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:22.266080 9515 heartbeater.cc:511] Master 127.29.233.190:41939 requested a full tablet report, sending...
I20260810 19:33:22.267869 9015 ts_manager.cc:194] Registered new tserver with Master: 1fcea8438a394cba95c5fdea7f6398b1 (127.29.233.131:42641)
I20260810 19:33:22.278586 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 19:33:22.285552 9226 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:33:22.283680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.29.233.190:41939)
0810 19:33:22.283887 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:22.283893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:22.284456 (+ 563us) server_negotiation.cc:407] Connection header received
0810 19:33:22.284633 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:22.284639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:22.284710 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:22.284827 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:22.285231 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43862: BlockingRecv error: recv got EOF from 127.0.0.1:43862 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:33:22.286091 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 9080
I20260810 19:33:22.299189 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:22.287605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.29.233.129:39609)
0810 19:33:22.288368 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:22.288381 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:22.296164 (+ 7783us) server_negotiation.cc:407] Connection header received
0810 19:33:22.296326 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:22.296339 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:22.296651 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:22.296845 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:22.298136 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:22.298147 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:22.298165 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:22.298186 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:22.298254 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:22.298279 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:22.298554 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:22.298635 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 19:33:22.298748 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":373,"threads_started":1}
I20260810 19:33:22.300498 9219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:22.303639 9224 heartbeater.cc:503] Master 127.29.233.190:41939 was elected leader, sending a full tablet report...
I20260810 19:33:22.385589 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 9080
I20260810 19:33:22.405272 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 9228
I20260810 19:33:22.409471 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:22.406429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.29.233.130:43183)
0810 19:33:22.407042 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:22.407059 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:22.407088 (+ 29us) server_negotiation.cc:407] Connection header received
0810 19:33:22.407159 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:22.407167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:22.407404 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:22.407558 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:22.408718 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:22.408725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:22.408737 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:22.408751 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:22.408798 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:22.408811 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:22.408929 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:22.409055 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 19:33:22.409149 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":319,"threads_started":1}
I20260810 19:33:22.410948 9362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:22.496115 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 9228
I20260810 19:33:22.519171 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 9374
I20260810 19:33:22.523387 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:22.520265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56498 (local address 127.29.233.131:42641)
0810 19:33:22.520965 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:22.520972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:22.521005 (+ 33us) server_negotiation.cc:407] Connection header received
0810 19:33:22.521070 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:22.521076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:22.521310 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:22.521467 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:22.522580 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:22.522588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:22.522601 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:33:22.522620 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:22.522679 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:33:22.522700 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:22.522858 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:22.522996 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 19:33:22.523091 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":355,"threads_started":1}
I20260810 19:33:22.524711 9510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:22.609493 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 9374
I20260810 19:33:22.629348 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 8977
I20260810 19:33:22.630815 9048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:22.737888 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 8977
2026-08-10T19:33:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2566 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:33:22 dist-test-slave-tm1g krb5kdc[9535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:33:22 dist-test-slave-tm1g krb5kdc[9535](info): set up 2 sockets
Aug 10 19:33:22 dist-test-slave-tm1g krb5kdc[9535](info): commencing operation
krb5kdc: starting...
W20260810 19:33:23.957068 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:33:24 dist-test-slave-tm1g krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:33:24.270202 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43935
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:33:24.621735 9550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:24.622115 9550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:24.622291 9550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:24.629796 9550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:33:24.629961 9550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:24.630079 9550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:24.630188 9550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:33:24.630280 9550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:33:24.640540 9550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:43935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:24.642100 9550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:24.643555 9550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:24.656206 9556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:24.656260 9559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:24.656179 9557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:24.657126 9550 server_base.cc:1034] running on GCE node
I20260810 19:33:24.658016 9550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:24.659584 9550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:24.660826 9550 hybrid_clock.cc:648] HybridClock initialized: now 1786390404660775 us; error 77 us; skew 500 ppm
I20260810 19:33:24.661185 9550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:24 dist-test-slave-tm1g krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390404, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:24.670672 9550 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:33:24.672281 9550 webserver.cc:460] Webserver started at http://127.29.233.190:42017/ using document root <none> and password file <none>
I20260810 19:33:24.673048 9550 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:24.673203 9550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:24.673538 9550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:24.676734 9550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "6abeb377f49345c890153aa3f3afeb1f"
format_stamp: "Formatted at 2026-08-10 19:33:24 on dist-test-slave-tm1g"
server_key: "bf01a451478f03bfa2d8a50959691261"
server_key_iv: "7177c0c9042ce9e5fef24cd068e123ae"
server_key_version: "encryptionkey@0"
I20260810 19:33:24.677556 9550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "6abeb377f49345c890153aa3f3afeb1f"
format_stamp: "Formatted at 2026-08-10 19:33:24 on dist-test-slave-tm1g"
server_key: "bf01a451478f03bfa2d8a50959691261"
server_key_iv: "7177c0c9042ce9e5fef24cd068e123ae"
server_key_version: "encryptionkey@0"
I20260810 19:33:24.683054 9550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:33:24.686640 9566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:24.687974 9550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:33:24.688186 9550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "6abeb377f49345c890153aa3f3afeb1f"
format_stamp: "Formatted at 2026-08-10 19:33:24 on dist-test-slave-tm1g"
server_key: "bf01a451478f03bfa2d8a50959691261"
server_key_iv: "7177c0c9042ce9e5fef24cd068e123ae"
server_key_version: "encryptionkey@0"
I20260810 19:33:24.688406 9550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:24.701726 9550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:24.706202 9550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:24.706563 9550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:24.725914 9550 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:43935
I20260810 19:33:24.725952 9622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:43935 every 8 connection(s)
I20260810 19:33:24.727380 9550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:33:24.730481 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 9550
I20260810 19:33:24.730736 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:33:24.731619 30630 external_mini_cluster.cc:1428] Setting key 952b8e7b6da5299588f28f237343384b
I20260810 19:33:24.731765 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:33:24.745951 9624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f: Bootstrap starting.
Aug 10 19:33:24 dist-test-slave-tm1g krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:33:24.749490 9624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f: Neither blocks nor log segments found. Creating new log.
I20260810 19:33:24.750702 9624 log.cc:824] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:33:24.753615 9624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f: No bootstrap required, opened a new log
I20260810 19:33:24.755103 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:24.733829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59814 (local address 127.29.233.190:43935)
0810 19:33:24.734696 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:24.734711 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:24.734762 (+ 51us) server_negotiation.cc:407] Connection header received
0810 19:33:24.737022 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:24.737051 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:24.737621 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:24.738070 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:24.739805 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:24.741227 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:24.742582 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:24.743297 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:24.743551 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:24.747714 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:24.747722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:24.747735 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:24.747776 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:24.751052 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:24.751765 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:24.751772 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:24.751781 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:24.751943 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:24.752348 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:24.752358 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:24.752360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:24.752876 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:24.753217 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:24.753641 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 19:33:24.754065 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":350,"threads_started":1}
I20260810 19:33:24.761785 9624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb377f49345c890153aa3f3afeb1f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43935 } }
I20260810 19:33:24.762210 9624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:33:24.762368 9624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abeb377f49345c890153aa3f3afeb1f, State: Initialized, Role: FOLLOWER
I20260810 19:33:24.762943 9624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb377f49345c890153aa3f3afeb1f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43935 } }
I20260810 19:33:24.763151 9624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:33:24.763247 9624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:33:24.763376 9624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:33:24.765935 9624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb377f49345c890153aa3f3afeb1f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43935 } }
I20260810 19:33:24.766462 9624 leader_election.cc:302] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6abeb377f49345c890153aa3f3afeb1f; no voters:
I20260810 19:33:24.767066 9624 leader_election.cc:288] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:33:24.767210 9629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:33:24.767899 9629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [term 1 LEADER]: Becoming Leader. State: Replica: 6abeb377f49345c890153aa3f3afeb1f, State: Running, Role: LEADER
I20260810 19:33:24.768529 9629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb377f49345c890153aa3f3afeb1f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43935 } }
I20260810 19:33:24.769270 9624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:33:24.771714 9630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6abeb377f49345c890153aa3f3afeb1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb377f49345c890153aa3f3afeb1f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43935 } } }
I20260810 19:33:24.771703 9631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6abeb377f49345c890153aa3f3afeb1f. Latest consensus state: current_term: 1 leader_uuid: "6abeb377f49345c890153aa3f3afeb1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6abeb377f49345c890153aa3f3afeb1f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43935 } } }
I20260810 19:33:24.772130 9630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:24.772179 9631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f [sys.catalog]: This master's current role is: LEADER
I20260810 19:33:24.775192 9637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:33:24.780905 9637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:33:24.790050 9637 catalog_manager.cc:1347] Generated new cluster ID: d3d490d9941741fb8273eeae6b053f4b
I20260810 19:33:24.790185 9637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:33:24.808349 9637 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:33:24.809928 9637 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:33:24.819438 9637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f: Generated new TSK 0
I20260810 19:33:24.820603 9637 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:33:24.911243 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43935
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:33:25.274955 9652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:25.275309 9652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:25.275489 9652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:25.283421 9652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:25.283538 9652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:25.283736 9652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:33:25.294535 9652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:25.295936 9652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:25.297384 9652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:25.311125 9658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:25.311205 9662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:25.311149 9659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:25.311797 9652 server_base.cc:1034] running on GCE node
I20260810 19:33:25.312541 9652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:25.313504 9652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:25.314754 9652 hybrid_clock.cc:648] HybridClock initialized: now 1786390405314701 us; error 62 us; skew 500 ppm
I20260810 19:33:25.315104 9652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:25 dist-test-slave-tm1g krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:25.318380 9652 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:33:25.320003 9652 webserver.cc:460] Webserver started at http://127.29.233.129:39799/ using document root <none> and password file <none>
I20260810 19:33:25.320739 9652 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:25.320883 9652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:25.321228 9652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:25.324398 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "4dbfbc610bb3475d9598516030f3776f"
format_stamp: "Formatted at 2026-08-10 19:33:25 on dist-test-slave-tm1g"
server_key: "b06a64413e5f4c54cd61b872f402d91d"
server_key_iv: "c2957c99f876208f9d47af515a203f0f"
server_key_version: "encryptionkey@0"
I20260810 19:33:25.325343 9652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "4dbfbc610bb3475d9598516030f3776f"
format_stamp: "Formatted at 2026-08-10 19:33:25 on dist-test-slave-tm1g"
server_key: "b06a64413e5f4c54cd61b872f402d91d"
server_key_iv: "c2957c99f876208f9d47af515a203f0f"
server_key_version: "encryptionkey@0"
I20260810 19:33:25.331529 9652 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 19:33:25.335363 9669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:25.336848 9652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:33:25.337045 9652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "4dbfbc610bb3475d9598516030f3776f"
format_stamp: "Formatted at 2026-08-10 19:33:25 on dist-test-slave-tm1g"
server_key: "b06a64413e5f4c54cd61b872f402d91d"
server_key_iv: "c2957c99f876208f9d47af515a203f0f"
server_key_version: "encryptionkey@0"
I20260810 19:33:25.337266 9652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:25.360080 9652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:25.364864 9652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:25.365213 9652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:25.366411 9652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:25.367884 9652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:25.368002 9652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:25.368152 9652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:25.368230 9652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:25.403501 9652 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39891
I20260810 19:33:25.403599 9793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39891 every 8 connection(s)
I20260810 19:33:25.404863 9652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:33:25.414867 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 9652
I20260810 19:33:25.415124 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:33:25.415980 30630 external_mini_cluster.cc:1428] Setting key 9a404e6b1475667ee74b9258de28f337
Aug 10 19:33:25 dist-test-slave-tm1g krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:33:25.426118 9796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:25.407606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:40593 (local address 127.29.233.190:43935)
0810 19:33:25.408104 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:25.408109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:25.408887 (+ 778us) server_negotiation.cc:407] Connection header received
0810 19:33:25.410807 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:25.410813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:25.410918 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:25.411050 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:25.412616 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.413349 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.415119 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.415786 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.415936 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:25.420019 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:25.420024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:25.420028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:25.420065 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:25.422894 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:25.423696 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:25.423707 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:25.423710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:25.423830 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:25.424287 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:25.424296 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:25.424299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:25.424570 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:25.424754 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:25.425482 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 19:33:25.425728 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":272,"threads_started":1}
I20260810 19:33:25.427151 9797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:25.407984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:43935 (local address 127.29.233.129:40593)
0810 19:33:25.408712 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:25.408751 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:25.410522 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:25.411291 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:25.411300 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:25.411824 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:25.412415 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:25.412433 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.413528 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.413533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:25.414902 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:25.414915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.416064 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.416075 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:25.416330 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:25.417059 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:25.417084 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:25.419767 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:25.423071 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:33:25.423080 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:33:25.423098 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:33:25.423553 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:33:25.423949 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:33:25.423953 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:33:25.423957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:33:25.424156 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:33:25.424710 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:25.424718 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:25.425082 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 19:33:25.425366 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 19:33:25.425942 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":274,"threads_started":1}
I20260810 19:33:25.428860 9794 heartbeater.cc:347] Connected to a master server at 127.29.233.190:43935
I20260810 19:33:25.429186 9794 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:25.429951 9794 heartbeater.cc:511] Master 127.29.233.190:43935 requested a full tablet report, sending...
I20260810 19:33:25.432190 9583 ts_manager.cc:194] Registered new tserver with Master: 4dbfbc610bb3475d9598516030f3776f (127.29.233.129:39891)
I20260810 19:33:25.433684 9583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:40593
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:33:25.487093 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43935
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:33:25.853012 9800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:25.853365 9800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:25.853526 9800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:25.861110 9800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:25.861272 9800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:25.861456 9800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:33:25.871779 9800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:25.873234 9800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:25.874789 9800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:25.888689 9805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:25.888741 9808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:25.888780 9806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:25.889627 9800 server_base.cc:1034] running on GCE node
I20260810 19:33:25.890286 9800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:25.891283 9800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:25.892525 9800 hybrid_clock.cc:648] HybridClock initialized: now 1786390405892482 us; error 39 us; skew 500 ppm
I20260810 19:33:25.892820 9800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:25 dist-test-slave-tm1g krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:25.896191 9800 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:33:25.897732 9800 webserver.cc:460] Webserver started at http://127.29.233.130:35263/ using document root <none> and password file <none>
I20260810 19:33:25.898566 9800 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:25.898736 9800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:25.899089 9800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:25.902062 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "bf7dba2a6e094ce6a5e6a2e041d02668"
format_stamp: "Formatted at 2026-08-10 19:33:25 on dist-test-slave-tm1g"
server_key: "f152e260dd7a9e61f91448eb336aa26d"
server_key_iv: "15a5f8f89628f6162577280ecbfaa77e"
server_key_version: "encryptionkey@0"
I20260810 19:33:25.902935 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "bf7dba2a6e094ce6a5e6a2e041d02668"
format_stamp: "Formatted at 2026-08-10 19:33:25 on dist-test-slave-tm1g"
server_key: "f152e260dd7a9e61f91448eb336aa26d"
server_key_iv: "15a5f8f89628f6162577280ecbfaa77e"
server_key_version: "encryptionkey@0"
I20260810 19:33:25.908321 9800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:33:25.911937 9816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:25.913295 9800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:33:25.913509 9800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "bf7dba2a6e094ce6a5e6a2e041d02668"
format_stamp: "Formatted at 2026-08-10 19:33:25 on dist-test-slave-tm1g"
server_key: "f152e260dd7a9e61f91448eb336aa26d"
server_key_iv: "15a5f8f89628f6162577280ecbfaa77e"
server_key_version: "encryptionkey@0"
I20260810 19:33:25.913717 9800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:25.942203 9800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:25.946388 9800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:25.946710 9800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:25.947679 9800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:25.949015 9800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:25.949120 9800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:25.949272 9800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:25.949344 9800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:25.981807 9800 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:41369
I20260810 19:33:25.981853 9943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:41369 every 8 connection(s)
I20260810 19:33:25.983175 9800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:33:25.991243 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 9800
I20260810 19:33:25.991487 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:33:25.991976 30630 external_mini_cluster.cc:1428] Setting key db78c84af750b44bd33e62c119408847
Aug 10 19:33:25 dist-test-slave-tm1g krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:33:26.005617 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:25.986043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:35491 (local address 127.29.233.190:43935)
0810 19:33:25.986582 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:25.986587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:25.987376 (+ 789us) server_negotiation.cc:407] Connection header received
0810 19:33:25.989206 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:25.989212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:25.989317 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:25.989442 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:25.991035 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.991997 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.993840 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.994287 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.994499 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:25.998381 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:25.998390 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:25.998398 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:25.998454 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:26.002089 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.002935 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.002942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.002946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.003113 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.003543 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.003550 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.003556 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.003960 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:26.004132 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.004820 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.005087 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":303,"threads_started":1}
I20260810 19:33:26.006481 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:25.986508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:43935 (local address 127.29.233.130:35491)
0810 19:33:25.987214 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:25.987253 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:25.988923 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:25.989685 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:25.989694 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:25.990223 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:25.990855 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:25.990871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.992153 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.992160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:25.993683 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:25.993701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:25.994579 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:25.994586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:25.994845 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:25.995568 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:25.995592 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:25.998090 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:26.002270 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:33:26.002278 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:33:26.002292 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:33:26.002810 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:33:26.003226 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:33:26.003229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:33:26.003231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:33:26.003423 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:33:26.004080 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:26.004086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:26.004411 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 19:33:26.004673 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 19:33:26.005093 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":276,"threads_started":1}
I20260810 19:33:26.008095 9945 heartbeater.cc:347] Connected to a master server at 127.29.233.190:43935
I20260810 19:33:26.008433 9945 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:26.009322 9945 heartbeater.cc:511] Master 127.29.233.190:43935 requested a full tablet report, sending...
I20260810 19:33:26.011027 9583 ts_manager.cc:194] Registered new tserver with Master: bf7dba2a6e094ce6a5e6a2e041d02668 (127.29.233.130:41369)
I20260810 19:33:26.011893 9583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:35491
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:33:26.061928 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43935
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:33:26.424568 9953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:33:26.424896 9953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:33:26.425068 9953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:33:26.432331 9953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:33:26.432497 9953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:33:26.432673 9953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:33:26.436715 9794 heartbeater.cc:503] Master 127.29.233.190:43935 was elected leader, sending a full tablet report...
I20260810 19:33:26.443495 9953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:33:26.444875 9953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:33:26.446223 9953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:33:26.460134 9958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:26.460162 9959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:33:26.460043 9961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:33:26.461119 9953 server_base.cc:1034] running on GCE node
I20260810 19:33:26.461748 9953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:33:26.462890 9953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:33:26.464099 9953 hybrid_clock.cc:648] HybridClock initialized: now 1786390406464034 us; error 53 us; skew 500 ppm
I20260810 19:33:26.464438 9953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:33:26 dist-test-slave-tm1g krb5kdc[9535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:33:26.467741 9953 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:33:26.469259 9953 webserver.cc:460] Webserver started at http://127.29.233.131:35811/ using document root <none> and password file <none>
I20260810 19:33:26.469997 9953 fs_manager.cc:359] Metadata directory not provided
I20260810 19:33:26.470134 9953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:33:26.470531 9953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:33:26.473544 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "24b6435c88e8457daf5108993735685a"
format_stamp: "Formatted at 2026-08-10 19:33:26 on dist-test-slave-tm1g"
server_key: "53389b74d838b8800a1ad82250662840"
server_key_iv: "2bedf818fd92f4ed5f9ab16d39895b42"
server_key_version: "encryptionkey@0"
I20260810 19:33:26.474458 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "24b6435c88e8457daf5108993735685a"
format_stamp: "Formatted at 2026-08-10 19:33:26 on dist-test-slave-tm1g"
server_key: "53389b74d838b8800a1ad82250662840"
server_key_iv: "2bedf818fd92f4ed5f9ab16d39895b42"
server_key_version: "encryptionkey@0"
I20260810 19:33:26.479792 9953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:33:26.483299 9969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:26.484894 9953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:33:26.485071 9953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "24b6435c88e8457daf5108993735685a"
format_stamp: "Formatted at 2026-08-10 19:33:26 on dist-test-slave-tm1g"
server_key: "53389b74d838b8800a1ad82250662840"
server_key_iv: "2bedf818fd92f4ed5f9ab16d39895b42"
server_key_version: "encryptionkey@0"
I20260810 19:33:26.485318 9953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:33:26.499173 9953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:33:26.503567 9953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:33:26.503887 9953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:33:26.504861 9953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:33:26.506474 9953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:33:26.506574 9953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:26.506727 9953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:33:26.506804 9953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:33:26.541354 9953 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:44861
I20260810 19:33:26.541391 10101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:44861 every 8 connection(s)
I20260810 19:33:26.542882 9953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:33:26.552070 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 9953
I20260810 19:33:26.552289 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:33:26.552768 30630 external_mini_cluster.cc:1428] Setting key 7912b15ef21292aa2030f2087a4c026a
Aug 10 19:33:26 dist-test-slave-tm1g krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:33:26.563745 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.545499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:33053 (local address 127.29.233.190:43935)
0810 19:33:26.545967 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.545971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.547056 (+ 1085us) server_negotiation.cc:407] Connection header received
0810 19:33:26.549133 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.549139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.549256 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.549382 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:26.550964 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.551780 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.553792 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.554404 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.554569 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.558226 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:26.558231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:26.558234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:26.558269 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:26.560871 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.561622 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.561625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.561627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.561748 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.562143 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.562147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.562149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.562496 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:26.562614 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.563282 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.563450 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":216,"threads_started":1}
I20260810 19:33:26.564728 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.546069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:43935 (local address 127.29.233.131:33053)
0810 19:33:26.546893 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:26.546932 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:26.548816 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:26.549632 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:26.549641 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:26.550148 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:33:26.550781 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.550798 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.551954 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.551959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.553437 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.553453 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.554637 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.554642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.554956 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.555693 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:33:26.555726 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:33:26.558029 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:33:26.561039 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:33:26.561046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:33:26.561059 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:33:26.561515 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:33:26.561846 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:33:26.561849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:33:26.561851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:33:26.562043 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:33:26.562619 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:33:26.562625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:33:26.562957 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 19:33:26.563194 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 19:33:26.563598 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":363,"threads_started":1}
I20260810 19:33:26.566145 10103 heartbeater.cc:347] Connected to a master server at 127.29.233.190:43935
I20260810 19:33:26.566486 10103 heartbeater.cc:464] Registering TS with master...
I20260810 19:33:26.567421 10103 heartbeater.cc:511] Master 127.29.233.190:43935 requested a full tablet report, sending...
I20260810 19:33:26.568917 9583 ts_manager.cc:194] Registered new tserver with Master: 24b6435c88e8457daf5108993735685a (127.29.233.131:44861)
I20260810 19:33:26.569985 9583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:33053
I20260810 19:33:26.580828 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:33:26.598533 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.585556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.29.233.190:43935)
0810 19:33:26.585727 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.585732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.586268 (+ 536us) server_negotiation.cc:407] Connection header received
0810 19:33:26.586549 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.586553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.586633 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.586740 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:26.588195 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.588940 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.590169 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.590667 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.590804 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.593224 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:26.593231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:26.593240 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:26.593274 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:26.595790 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.596498 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.596503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.596507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.596638 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.597036 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.597041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.597044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.597363 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:26.597641 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.597955 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.598178 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:33:26.602635 9583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:33:26.609582 9583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:33:26.634186 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.622370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:41369 (local address 127.0.0.1:55094)
0810 19:33:26.623602 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:26.623659 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:26.623871 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:26.626170 (+ 2299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:26.626180 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:26.626216 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:33:26.627228 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.627244 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.628660 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.628665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.631283 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.631296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.633028 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.633035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.633255 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.633321 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 19:33:26.633444 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 19:33:26.633663 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":64,"thread_start_us":2814,"threads_started":1}
I20260810 19:33:26.634996 10119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.622551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.29.233.130:41369)
0810 19:33:26.625196 (+ 2645us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.625203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.625230 (+ 27us) server_negotiation.cc:407] Connection header received
0810 19:33:26.625358 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.625366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.625610 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.625838 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:33:26.627468 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.628511 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.631448 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.632780 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.632965 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.633139 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.634484 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.634664 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2448,"thread_start_us":2293,"threads_started":1}
I20260810 19:33:26.635301 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.623394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.29.233.131:44861)
0810 19:33:26.624663 (+ 1269us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.624675 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.625609 (+ 934us) server_negotiation.cc:407] Connection header received
0810 19:33:26.625793 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.625800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.626018 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.626472 (+ 454us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:33:26.627231 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.629221 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.631936 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.633312 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.633521 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.633686 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.633949 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.634108 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":903,"threads_started":1}
I20260810 19:33:26.635609 10118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.622766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:44861 (local address 127.0.0.1:52564)
0810 19:33:26.625478 (+ 2712us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:26.625497 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:26.625627 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:26.626223 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:26.626227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:26.626248 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:33:26.626931 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.626965 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.629369 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.629377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.631768 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.631783 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.633555 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.633562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.633754 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.633778 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:33:26.633898 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 19:33:26.634040 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2595,"mutex_wait_us":3,"thread_start_us":424,"threads_started":1}
I20260810 19:33:26.637013 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.624753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39891 (local address 127.0.0.1:58328)
0810 19:33:26.626180 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:26.626199 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:26.626334 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:26.629722 (+ 3388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:26.629728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:26.629761 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:33:26.630348 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.630361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.632215 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.632223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.634079 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.634097 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.636254 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.636259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.636372 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.636394 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 19:33:26.636515 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 19:33:26.636643 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1275,"mutex_wait_us":27,"thread_start_us":1131,"threads_started":1}
I20260810 19:33:26.637064 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.625494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.29.233.129:39891)
0810 19:33:26.629145 (+ 3651us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.629154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.629180 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:33:26.629282 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.629289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.629513 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.629709 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:33:26.630568 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.632084 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.634296 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.635762 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.635961 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.636120 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.636568 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.636714 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3488,"thread_start_us":3308,"threads_started":1}
I20260810 19:33:26.639235 9870 tablet_service.cc:1467] Processing CreateTablet for tablet b67c442fc58744d5801aff19e771c9d9 (DEFAULT_TABLE table=test-table [id=d600a10f7a79457393abd7e0be4d62cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:33:26.639235 10023 tablet_service.cc:1467] Processing CreateTablet for tablet b67c442fc58744d5801aff19e771c9d9 (DEFAULT_TABLE table=test-table [id=d600a10f7a79457393abd7e0be4d62cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:33:26.639981 9721 tablet_service.cc:1467] Processing CreateTablet for tablet b67c442fc58744d5801aff19e771c9d9 (DEFAULT_TABLE table=test-table [id=d600a10f7a79457393abd7e0be4d62cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:33:26.640826 9870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67c442fc58744d5801aff19e771c9d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:33:26.640815 10023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67c442fc58744d5801aff19e771c9d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:33:26.641319 9721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67c442fc58744d5801aff19e771c9d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:33:26.654657 10125 tablet_bootstrap.cc:492] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Bootstrap starting.
I20260810 19:33:26.656194 10126 tablet_bootstrap.cc:492] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Bootstrap starting.
I20260810 19:33:26.656989 10127 tablet_bootstrap.cc:492] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Bootstrap starting.
I20260810 19:33:26.658376 10125 tablet_bootstrap.cc:654] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Neither blocks nor log segments found. Creating new log.
I20260810 19:33:26.659754 10125 log.cc:824] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:33:26.660768 10127 tablet_bootstrap.cc:654] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Neither blocks nor log segments found. Creating new log.
I20260810 19:33:26.661247 10126 tablet_bootstrap.cc:654] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Neither blocks nor log segments found. Creating new log.
I20260810 19:33:26.662039 10127 log.cc:824] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Log is configured to *not* fsync() on all Append() calls
I20260810 19:33:26.662524 10126 log.cc:824] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Log is configured to *not* fsync() on all Append() calls
I20260810 19:33:26.663478 10125 tablet_bootstrap.cc:492] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: No bootstrap required, opened a new log
I20260810 19:33:26.663789 10125 ts_tablet_manager.cc:1397] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 19:33:26.664844 10127 tablet_bootstrap.cc:492] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: No bootstrap required, opened a new log
I20260810 19:33:26.665102 10127 ts_tablet_manager.cc:1397] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260810 19:33:26.667126 10126 tablet_bootstrap.cc:492] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: No bootstrap required, opened a new log
I20260810 19:33:26.667383 10126 ts_tablet_manager.cc:1397] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260810 19:33:26.672806 10127 raft_consensus.cc:348] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.673198 10127 raft_consensus.cc:374] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:33:26.673292 10127 raft_consensus.cc:729] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7dba2a6e094ce6a5e6a2e041d02668, State: Initialized, Role: FOLLOWER
I20260810 19:33:26.673945 10127 consensus_queue.cc:260] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.674594 10125 raft_consensus.cc:348] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.675052 10125 raft_consensus.cc:374] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:33:26.675155 10125 raft_consensus.cc:729] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbfbc610bb3475d9598516030f3776f, State: Initialized, Role: FOLLOWER
I20260810 19:33:26.675485 9945 heartbeater.cc:503] Master 127.29.233.190:43935 was elected leader, sending a full tablet report...
I20260810 19:33:26.675953 10127 ts_tablet_manager.cc:1428] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 19:33:26.675769 10125 consensus_queue.cc:260] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.677464 10125 ts_tablet_manager.cc:1428] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260810 19:33:26.679356 10126 raft_consensus.cc:348] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.679759 10126 raft_consensus.cc:374] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:33:26.679884 10126 raft_consensus.cc:729] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b6435c88e8457daf5108993735685a, State: Initialized, Role: FOLLOWER
I20260810 19:33:26.680501 10126 consensus_queue.cc:260] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.683517 10103 heartbeater.cc:503] Master 127.29.233.190:43935 was elected leader, sending a full tablet report...
I20260810 19:33:26.684240 10126 ts_tablet_manager.cc:1428] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Time spent starting tablet: real 0.017s user 0.009s sys 0.006s
W20260810 19:33:26.735288 9948 tablet.cc:2369] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:33:26.760159 10134 raft_consensus.cc:482] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:33:26.760636 10134 raft_consensus.cc:504] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.762516 10134 leader_election.cc:288] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf7dba2a6e094ce6a5e6a2e041d02668 (127.29.233.130:41369), 4dbfbc610bb3475d9598516030f3776f (127.29.233.129:39891)
I20260810 19:33:26.769107 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.763244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:38699 (local address 127.29.233.129:39891)
0810 19:33:26.763463 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.763470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.763491 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:33:26.763573 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.763578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.763661 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.763778 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:33:26.764220 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.765233 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.767087 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.768256 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.768377 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.768430 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.768767 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.768898 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:33:26.769172 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.762844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39891 (local address 127.29.233.131:38699)
0810 19:33:26.763256 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:26.763284 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:26.763446 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:26.763802 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:26.763806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:26.763829 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:33:26.764074 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.764086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.765382 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.765387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.766923 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.766935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.768407 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.768413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.768603 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.768638 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 19:33:26.768740 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 19:33:26.768880 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244}
I20260810 19:33:26.770212 10119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.762998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:43295 (local address 127.29.233.130:41369)
0810 19:33:26.763233 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.763241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.763928 (+ 687us) server_negotiation.cc:407] Connection header received
0810 19:33:26.764100 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.764106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.764196 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.764276 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:33:26.764913 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.765862 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.768073 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.769223 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.769370 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.769453 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.769874 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.770026 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 19:33:26.770356 10135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.762901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:41369 (local address 127.29.233.131:43295)
0810 19:33:26.763805 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0810 19:33:26.763823 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:33:26.763960 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:33:26.764322 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:33:26.764327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:33:26.764353 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:33:26.764762 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.764774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.766049 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.766053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.767898 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:33:26.767913 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.769532 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.769537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:33:26.769670 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.769693 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:33:26.769770 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 19:33:26.769917 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":3,"thread_start_us":541,"threads_started":1}
I20260810 19:33:26.770479 9742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67c442fc58744d5801aff19e771c9d9" candidate_uuid: "24b6435c88e8457daf5108993735685a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbfbc610bb3475d9598516030f3776f" is_pre_election: true
I20260810 19:33:26.771127 9742 raft_consensus.cc:2454] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24b6435c88e8457daf5108993735685a in term 0.
I20260810 19:33:26.771328 9890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67c442fc58744d5801aff19e771c9d9" candidate_uuid: "24b6435c88e8457daf5108993735685a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" is_pre_election: true
I20260810 19:33:26.771807 9970 leader_election.cc:302] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24b6435c88e8457daf5108993735685a, 4dbfbc610bb3475d9598516030f3776f; no voters:
I20260810 19:33:26.771804 9890 raft_consensus.cc:2454] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24b6435c88e8457daf5108993735685a in term 0.
I20260810 19:33:26.772185 10134 raft_consensus.cc:2781] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:33:26.772357 10134 raft_consensus.cc:482] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:33:26.772477 10134 raft_consensus.cc:3037] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:33:26.774843 10134 raft_consensus.cc:504] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.775523 10134 leader_election.cc:288] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [CANDIDATE]: Term 1 election: Requested vote from peers bf7dba2a6e094ce6a5e6a2e041d02668 (127.29.233.130:41369), 4dbfbc610bb3475d9598516030f3776f (127.29.233.129:39891)
I20260810 19:33:26.775923 9890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67c442fc58744d5801aff19e771c9d9" candidate_uuid: "24b6435c88e8457daf5108993735685a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668"
I20260810 19:33:26.776021 9742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67c442fc58744d5801aff19e771c9d9" candidate_uuid: "24b6435c88e8457daf5108993735685a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbfbc610bb3475d9598516030f3776f"
I20260810 19:33:26.776113 9890 raft_consensus.cc:3037] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:33:26.776263 9742 raft_consensus.cc:3037] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:33:26.778491 9890 raft_consensus.cc:2454] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24b6435c88e8457daf5108993735685a in term 1.
I20260810 19:33:26.778506 9742 raft_consensus.cc:2454] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24b6435c88e8457daf5108993735685a in term 1.
I20260810 19:33:26.778908 9973 leader_election.cc:302] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24b6435c88e8457daf5108993735685a, bf7dba2a6e094ce6a5e6a2e041d02668; no voters:
I20260810 19:33:26.779256 10134 raft_consensus.cc:2781] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:33:26.779714 10134 raft_consensus.cc:686] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 1 LEADER]: Becoming Leader. State: Replica: 24b6435c88e8457daf5108993735685a, State: Running, Role: LEADER
I20260810 19:33:26.780177 10134 consensus_queue.cc:237] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } }
I20260810 19:33:26.785030 9583 catalog_manager.cc:5495] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a reported cstate change: term changed from 0 to 1, leader changed from <none> to 24b6435c88e8457daf5108993735685a (127.29.233.131). New cstate: current_term: 1 leader_uuid: "24b6435c88e8457daf5108993735685a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24b6435c88e8457daf5108993735685a" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 44861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 } health_report { overall_health: UNKNOWN } } }
W20260810 19:33:26.794540 10104 tablet.cc:2369] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:33:26.835904 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.829754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.29.233.131:44861)
0810 19:33:26.829990 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.829996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.830016 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:33:26.830227 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.830232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.830384 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.830555 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:33:26.830984 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.832253 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.833618 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.834031 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.834169 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.834766 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:33:26.835304 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:33:26.835481 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.835536 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.835619 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 19:33:26.844205 9742 raft_consensus.cc:1261] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 1 FOLLOWER]: Refusing update from remote peer 24b6435c88e8457daf5108993735685a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:33:26.844265 9890 raft_consensus.cc:1261] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 1 FOLLOWER]: Refusing update from remote peer 24b6435c88e8457daf5108993735685a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:33:26.845206 10136 consensus_queue.cc:1035] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbfbc610bb3475d9598516030f3776f" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:33:26.845624 10134 consensus_queue.cc:1035] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf7dba2a6e094ce6a5e6a2e041d02668" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:33:26.870687 9582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:33:26.870995 9582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:33:26.874874 9582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6abeb377f49345c890153aa3f3afeb1f: Sending DeleteTablet for 3 replicas of tablet b67c442fc58744d5801aff19e771c9d9
I20260810 19:33:26.876089 9870 tablet_service.cc:1514] Processing DeleteTablet for tablet b67c442fc58744d5801aff19e771c9d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:33:26 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:55094
I20260810 19:33:26.876096 10023 tablet_service.cc:1514] Processing DeleteTablet for tablet b67c442fc58744d5801aff19e771c9d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:33:26 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52564
I20260810 19:33:26.876250 9721 tablet_service.cc:1514] Processing DeleteTablet for tablet b67c442fc58744d5801aff19e771c9d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:33:26 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:58328
I20260810 19:33:26.881273 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 9652
I20260810 19:33:26.889348 10153 tablet_replica.cc:330] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: stopping tablet replica
I20260810 19:33:26.889760 10151 tablet_replica.cc:330] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: stopping tablet replica
I20260810 19:33:26.890481 10153 raft_consensus.cc:2229] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:33:26.890785 10151 raft_consensus.cc:2229] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:33:26.891229 10153 raft_consensus.cc:2258] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:33:26.891305 10151 raft_consensus.cc:2258] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:33:26.893599 10151 ts_tablet_manager.cc:1905] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:33:26.893806 10153 ts_tablet_manager.cc:1905] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:33:26.900868 10151 ts_tablet_manager.cc:1918] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:33:26.901037 10151 log.cc:1196] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/b67c442fc58744d5801aff19e771c9d9
I20260810 19:33:26.901597 10151 ts_tablet_manager.cc:1939] T b67c442fc58744d5801aff19e771c9d9 P bf7dba2a6e094ce6a5e6a2e041d02668: Deleting consensus metadata
I20260810 19:33:26.902415 10153 ts_tablet_manager.cc:1918] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:33:26.902675 10153 log.cc:1196] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/b67c442fc58744d5801aff19e771c9d9
I20260810 19:33:26.903302 10153 ts_tablet_manager.cc:1939] T b67c442fc58744d5801aff19e771c9d9 P 4dbfbc610bb3475d9598516030f3776f: Deleting consensus metadata
I20260810 19:33:26.903774 9570 catalog_manager.cc:4841] TS bf7dba2a6e094ce6a5e6a2e041d02668 (127.29.233.130:41369): tablet b67c442fc58744d5801aff19e771c9d9 (table test-table [id=d600a10f7a79457393abd7e0be4d62cb]) successfully deleted
I20260810 19:33:26.906409 9567 catalog_manager.cc:4841] TS 4dbfbc610bb3475d9598516030f3776f (127.29.233.129:39891): tablet b67c442fc58744d5801aff19e771c9d9 (table test-table [id=d600a10f7a79457393abd7e0be4d62cb]) successfully deleted
I20260810 19:33:26.908430 10152 tablet_replica.cc:330] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: stopping tablet replica
I20260810 19:33:26.908972 10152 raft_consensus.cc:2229] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:33:26.910154 10152 raft_consensus.cc:2258] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:33:26.912526 10152 ts_tablet_manager.cc:1905] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 19:33:26 dist-test-slave-tm1g krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:33:26.920145 10152 ts_tablet_manager.cc:1918] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:33:26.920339 10152 log.cc:1196] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/b67c442fc58744d5801aff19e771c9d9
I20260810 19:33:26.920984 10152 ts_tablet_manager.cc:1939] T b67c442fc58744d5801aff19e771c9d9 P 24b6435c88e8457daf5108993735685a: Deleting consensus metadata
I20260810 19:33:26.921913 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:26.882787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.29.233.129:39891)
0810 19:33:26.882974 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:26.882980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:26.901533 (+ 18553us) server_negotiation.cc:407] Connection header received
0810 19:33:26.902042 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:26.902048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:26.902142 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:26.902235 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:26.904169 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.905387 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.907384 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:26.908198 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:26.908332 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:26.913871 (+ 5539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:26.913885 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:26.913902 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:26.913946 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:26.917968 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.919157 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.919166 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.919170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.919329 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:26.919796 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:26.919803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:26.919809 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:26.920297 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:26.920665 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:26.921297 (+ 632us) server_negotiation.cc:299] Negotiation successful
0810 19:33:26.921543 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 19:33:26.922936 9568 catalog_manager.cc:4841] TS 24b6435c88e8457daf5108993735685a (127.29.233.131:44861): tablet b67c442fc58744d5801aff19e771c9d9 (table test-table [id=d600a10f7a79457393abd7e0be4d62cb]) successfully deleted
I20260810 19:33:26.923475 9787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:27.064218 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 9652
I20260810 19:33:27.086375 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 9800
Aug 10 19:33:27 dist-test-slave-tm1g krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:33:27.103524 10119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:27.087286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55098 (local address 127.29.233.130:41369)
0810 19:33:27.087478 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:27.087484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:27.087504 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:33:27.087614 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:27.087620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:27.087694 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:27.087818 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:27.089124 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:27.089995 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:27.091554 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:27.091961 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:27.092132 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:27.097263 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:27.097278 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:27.097296 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:27.097331 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:27.100489 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:27.101413 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:27.101420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:27.101423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:27.101568 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:27.101960 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:27.101963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:27.101965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:27.102441 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:27.102696 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:27.103018 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 19:33:27.103184 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 19:33:27.104319 9937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:27.195803 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 9800
I20260810 19:33:27.223690 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 9953
Aug 10 19:33:27 dist-test-slave-tm1g krb5kdc[9535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:33:27.242113 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:33:27.224546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.29.233.131:44861)
0810 19:33:27.224724 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 19:33:27.224729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:33:27.224748 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:33:27.224954 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:33:27.224958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:33:27.225049 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:33:27.225134 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:33:27.227000 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:27.227809 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:27.229698 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:33:27.230160 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:33:27.230261 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:33:27.235764 (+ 5503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:33:27.235780 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:33:27.235802 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:33:27.235844 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:33:27.239176 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:27.239887 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:27.239894 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:27.239896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:27.240066 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:33:27.240463 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:33:27.240469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:33:27.240472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:33:27.240851 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:33:27.241233 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0810 19:33:27.241510 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 19:33:27.241735 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:33:27.242873 10092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:27.369369 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 9953
I20260810 19:33:27.393098 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 9550
I20260810 19:33:27.394268 9618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:33:27.507022 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 9550
2026-08-10T19:33:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4773 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:33:27 dist-test-slave-tm1g krb5kdc[10161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:33:27 dist-test-slave-tm1g krb5kdc[10161](info): set up 2 sockets
Aug 10 19:33:27 dist-test-slave-tm1g krb5kdc[10161](info): commencing operation
krb5kdc: starting...
W20260810 19:33:28.715276 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:33:28 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:33:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/rangeradmin_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/rangeradmin_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/rangerlookup_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/rangerlookup_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
I20260810 19:33:29.146870 30630 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskW7W1IE/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/postgres -l logfile start
2026-08-10 19:33:31.923 UTC [10191] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 19:33:31.923 UTC [10191] LOG: listening on IPv4 address "127.29.233.148", port 43165
2026-08-10 19:33:31.928 UTC [10191] LOG: listening on Unix socket "/tmp/.s.PGSQL.43165"
2026-08-10 19:33:31.954 UTC [10193] LOG: database system was shut down at 2026-08-10 19:33:30 UTC
2026-08-10 19:33:31.958 UTC [10191] LOG: database system is ready to accept connections
I20260810 19:33:32.991263 30630 mini_postgres.cc:96] Postgres bound to 43165
2026-08-10 19:33:33.018 UTC [10207] FATAL: database "slave" does not exist
127.29.233.148:43165 - accepting connections
I20260810 19:33:33.019235 30630 mini_ranger.cc:161] Starting Ranger...
I20260810 19:33:33.054808 30630 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 19:33:33.644110 30630 mini_ranger.cc:87] Created ranger Postgres database
I20260810 19:33:33.644260 30630 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-admin
2026-08-10 19:33:33,974 [I] DB FLAVOR :POSTGRES
2026-08-10 19:33:33,975 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 19:33:33,975 [I] Checking connection..
2026-08-10 19:33:33,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 19:33:34,248 [I] Checking connection passed.
2026-08-10 19:33:34,248 [I] --------- Verifying version history table ---------
2026-08-10 19:33:34,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 19:33:34,505 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 19:33:34,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 19:33:34,762 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 19:33:34,762 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 19:33:34,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 19:33:35.025 UTC [10279] WARNING: there is no transaction in progress
2026-08-10 19:33:35,033 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 19:33:35,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 19:33:35,317 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 19:33:35,317 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 19:33:35,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 19:33:35,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 19:33:35,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-10 19:33:36,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 19:33:36,380 [I] Table x_portal_user does not exist in database ranger
2026-08-10 19:33:36,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 19:33:36,651 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 19:33:36,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 19:33:36,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 19:33:37,191 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 19:33:37,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 19:33:38.249 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.266 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.283 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.299 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.316 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.338 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.397 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.404 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.419 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.431 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.441 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.456 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.466 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.472 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.479 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:38.486 UTC [10395] WARNING: there is no transaction in progress
2026-08-10 19:33:39,228 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 19:33:39,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 19:33:39,476 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 19:33:39,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 19:33:39,728 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 19:33:39,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 19:33:39,967 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 19:33:39,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 19:33:40,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 19:33:40,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-10 19:33:40,751 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 19:33:40,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 19:33:41,005 [I] DB_PATCHES have already been applied
I20260810 19:33:41.011438 30630 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-admin:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskW7W1IE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 19:33:41.011545 30630 mini_ranger.cc:193] Using host: 127.29.233.148
I20260810 19:33:41.035408 30630 mini_ranger.cc:227] Ranger admin URL: http://127.29.233.148:44089
Aug 10, 2026 7:33:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:33:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:33:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:33:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:33:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:33:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:33:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 19:33:42.248397 30630 mini_ranger.cc:160] Time spent starting Ranger: real 9.229s user 0.002s sys 0.070s
Aug 10, 2026 7:34:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:34:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 7:34:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:34:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:34:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 19:34:03.366576 30630 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/rangerkms_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/rangerkms_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.233.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.233.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 19:34:03 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390443, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 19:34:03.600072 30630 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 19:34:03.629428 30630 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 19:34:04.558012 30630 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 19:34:04.558193 30630 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms
I20260810 19:34:04.558339 30630 mini_ranger_kms.cc:227] Using postgres at 127.29.233.148:43165
2026-08-10 19:34:05,060 [I] DB FLAVOR :POSTGRES
2026-08-10 19:34:05,060 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 19:34:05,060 [I] Checking connection
2026-08-10 19:34:05,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 19:34:05,304 [I] connection success
2026-08-10 19:34:05,304 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 19:34:05,304 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 19:34:05,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 19:34:05,560 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 19:34:05,560 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 19:34:05,560 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 19:34:05,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:43165/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 19:34:05,832 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 19:34:06.326190 30630 mini_ranger_kms.cc:310] Created kms service
I20260810 19:34:06.527951 30630 mini_ranger_kms.cc:326] Created kudu user
I20260810 19:34:06.576270 30630 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 19:34:06 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390443, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:34:07.028882 30630 mini_ranger_kms.cc:384] Added ranger policy
I20260810 19:34:07.029312 30630 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms:/tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 19:34:07.029381 30630 mini_ranger_kms.cc:240] Using host: 127.29.233.148
I20260810 19:34:07.057982 30630 mini_ranger_kms.cc:276] Ranger KMS PID: 10597
I20260810 19:34:07.058182 30630 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.233.148:46751
Aug 10, 2026 7:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 19:34:09 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786390449, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.233.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:34:09.539506 30630 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.939s user 0.126s sys 0.423s
I20260810 19:34:09.539868 30630 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 19:34:09.540093 30630 mini_ranger_kms.cc:401] 127.29.233.148:46751/kms/v1/keys
Aug 10 19:34:09 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390449, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.233.148@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
Aug 10 19:34:10 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390449, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.233.148@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
Aug 10 19:34:12 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:34:12.827389 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:44091
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:44091
--ranger_config_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:46751/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:34:13.186421 10654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:34:13.186718 10654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:34:13.186810 10654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:34:13.194028 10654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:34:13.194139 10654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:34:13.194187 10654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:34:13.194242 10654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:34:13.194293 10654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:34:13.204717 10654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.233.148:46751/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:44091
--ranger_config_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:44091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:34:13.206053 10654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:34:13.207376 10654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:34:13.220410 10663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:34:13.220434 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:34:13.220683 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:34:13.221678 10654 server_base.cc:1034] running on GCE node
I20260810 19:34:13.222648 10654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:34:13.224198 10654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:34:13.225456 10654 hybrid_clock.cc:648] HybridClock initialized: now 1786390453225412 us; error 68 us; skew 500 ppm
I20260810 19:34:13.225904 10654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:34:13 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390453, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:34:13.232636 10654 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:34:13.234392 10654 webserver.cc:460] Webserver started at http://127.29.233.190:45655/ using document root <none> and password file <none>
I20260810 19:34:13.235105 10654 fs_manager.cc:359] Metadata directory not provided
I20260810 19:34:13.235255 10654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:34:13.235590 10654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:34:13 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390453, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:34:13.295215 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "0fa70e6de98846febdcb2a9842696070"
format_stamp: "Formatted at 2026-08-10 19:34:13 on dist-test-slave-tm1g"
server_key: "957f19e098f8dc99759472019b90c057"
server_key_iv: "cfd1c4a4d387d9382f154ebe7853af48"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:13.296329 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "0fa70e6de98846febdcb2a9842696070"
format_stamp: "Formatted at 2026-08-10 19:34:13 on dist-test-slave-tm1g"
server_key: "957f19e098f8dc99759472019b90c057"
server_key_iv: "cfd1c4a4d387d9382f154ebe7853af48"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:13.303753 10654 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260810 19:34:13.325536 10672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:13.327500 10654 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 19:34:13.327721 10654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "0fa70e6de98846febdcb2a9842696070"
format_stamp: "Formatted at 2026-08-10 19:34:13 on dist-test-slave-tm1g"
server_key: "957f19e098f8dc99759472019b90c057"
server_key_iv: "cfd1c4a4d387d9382f154ebe7853af48"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:13.328029 10654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:34:13.365104 10654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:34:13.368889 10654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:34:13.369159 10654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:34:13.388419 10654 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:44091
I20260810 19:34:13.388492 10730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:44091 every 8 connection(s)
I20260810 19:34:13.389810 10654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:34:13.393697 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 10654
I20260810 19:34:13.393935 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:34:13.394058 10731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:34:13.394877 30630 external_mini_cluster.cc:1428] Setting key bf5533cab2d2f6b35fbe582bb1baea7d
I20260810 19:34:13.407367 10731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070: Bootstrap starting.
Aug 10 19:34:13 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:34:13.411103 10731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070: Neither blocks nor log segments found. Creating new log.
I20260810 19:34:13.412214 10731 log.cc:824] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070: Log is configured to *not* fsync() on all Append() calls
I20260810 19:34:13.415972 10731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070: No bootstrap required, opened a new log
I20260810 19:34:13.417603 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:13.397129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.29.233.190:44091)
0810 19:34:13.397935 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:13.397947 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:13.397991 (+ 44us) server_negotiation.cc:407] Connection header received
0810 19:34:13.399661 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:13.399682 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:13.400063 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:13.400504 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:13.401829 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:13.403190 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:13.404489 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:13.405204 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:13.405438 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:13.409710 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:13.409719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:13.409733 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:13.409769 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:13.412905 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:13.413672 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:13.413684 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:13.413690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:13.413933 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:13.414379 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:13.414384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:13.414387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:13.415094 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:13.415431 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:13.415809 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 19:34:13.416223 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":361,"threads_started":1}
I20260810 19:34:13.424687 10731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa70e6de98846febdcb2a9842696070" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44091 } }
I20260810 19:34:13.425065 10731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:34:13.425143 10731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa70e6de98846febdcb2a9842696070, State: Initialized, Role: FOLLOWER
I20260810 19:34:13.425704 10731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa70e6de98846febdcb2a9842696070" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44091 } }
I20260810 19:34:13.425921 10731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:34:13.426023 10731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:34:13.426147 10731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:34:13.428293 10731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa70e6de98846febdcb2a9842696070" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44091 } }
I20260810 19:34:13.428761 10731 leader_election.cc:302] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fa70e6de98846febdcb2a9842696070; no voters:
I20260810 19:34:13.429374 10731 leader_election.cc:288] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:34:13.429538 10736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:34:13.430644 10736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [term 1 LEADER]: Becoming Leader. State: Replica: 0fa70e6de98846febdcb2a9842696070, State: Running, Role: LEADER
I20260810 19:34:13.431157 10736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa70e6de98846febdcb2a9842696070" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44091 } }
I20260810 19:34:13.431735 10731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:34:13.433884 10738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fa70e6de98846febdcb2a9842696070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa70e6de98846febdcb2a9842696070" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44091 } } }
I20260810 19:34:13.434250 10738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [sys.catalog]: This master's current role is: LEADER
I20260810 19:34:13.435307 10742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:34:13.435113 10739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fa70e6de98846febdcb2a9842696070. Latest consensus state: current_term: 1 leader_uuid: "0fa70e6de98846febdcb2a9842696070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa70e6de98846febdcb2a9842696070" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44091 } } }
I20260810 19:34:13.435521 10739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070 [sys.catalog]: This master's current role is: LEADER
I20260810 19:34:13.441602 10742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:34:13.442588 10731 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 19:34:13.462500 10742 catalog_manager.cc:1347] Generated new cluster ID: 93900c32ba4d42feb5137ba6c15b24ec
I20260810 19:34:13.462733 10742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:34:13.498572 10742 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:34:13.500967 10742 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:34:13.516010 10742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070: Generated new TSK 0
I20260810 19:34:13.517227 10742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:34:14.299438 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:46751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44091
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:34:14 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786390454, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 19:34:14.722015 10782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:34:14.722473 10782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:34:14.722643 10782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:34:14.734002 10782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:34:14.734151 10782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:34:14.734421 10782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:34:14.750455 10782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.233.148:46751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:34:14.752207 10782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:34:14.753955 10782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:34:14.770761 10792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:34:14.771795 10794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:34:14.776957 10782 server_base.cc:1034] running on GCE node
W20260810 19:34:14.777333 10796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:34:14.778172 10782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:34:14.779452 10782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:34:14.780711 10782 hybrid_clock.cc:648] HybridClock initialized: now 1786390454780643 us; error 53 us; skew 500 ppm
I20260810 19:34:14.781113 10782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:34:14 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390454, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:34:14.785480 10782 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:34:14.787670 10782 webserver.cc:460] Webserver started at http://127.29.233.129:39639/ using document root <none> and password file <none>
I20260810 19:34:14.788774 10782 fs_manager.cc:359] Metadata directory not provided
I20260810 19:34:14.788934 10782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:34:14.789359 10782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:34:14 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390454, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:34:14.815343 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "38f61de586ac46a9be16d2ae149c3698"
format_stamp: "Formatted at 2026-08-10 19:34:14 on dist-test-slave-tm1g"
server_key: "93e40794bd9a42cf4fbef79707792cfb"
server_key_iv: "0331066f5cf4772021907397f8ab6e39"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:14.816569 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "38f61de586ac46a9be16d2ae149c3698"
format_stamp: "Formatted at 2026-08-10 19:34:14 on dist-test-slave-tm1g"
server_key: "93e40794bd9a42cf4fbef79707792cfb"
server_key_iv: "0331066f5cf4772021907397f8ab6e39"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:14.832959 10782 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.011s sys 0.003s
I20260810 19:34:14.866998 10805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:14.871860 10782 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.008s sys 0.004s
I20260810 19:34:14.872089 10782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "38f61de586ac46a9be16d2ae149c3698"
format_stamp: "Formatted at 2026-08-10 19:34:14 on dist-test-slave-tm1g"
server_key: "93e40794bd9a42cf4fbef79707792cfb"
server_key_iv: "0331066f5cf4772021907397f8ab6e39"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:14.872406 10782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:34:14.933183 10782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:34:14.938922 10782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:34:14.939435 10782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:34:14.950771 10782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:34:14.960640 10782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:34:14.960819 10782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 19:34:14.961048 10782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:34:14.961181 10782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:15.033067 10782 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:33707
I20260810 19:34:15.033090 10929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:33707 every 8 connection(s)
I20260810 19:34:15.034822 10782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:34:15.036187 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 10782
I20260810 19:34:15.036402 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:34:15.037217 30630 external_mini_cluster.cc:1428] Setting key b9ce2dbe97b068e56594ddbd2d5306d1
Aug 10 19:34:15 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390454, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
Aug 10 19:34:15 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390454, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:34:15.092619 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:15.038624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:54001 (local address 127.29.233.190:44091)
0810 19:34:15.068499 (+ 29875us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:15.068506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:15.068533 (+ 27us) server_negotiation.cc:407] Connection header received
0810 19:34:15.069841 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:15.069847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:15.069972 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:15.070126 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:15.072640 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.073385 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.075309 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.076069 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.076186 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:15.082215 (+ 6029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:15.082225 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:15.082233 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:15.082289 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:15.086054 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:15.088814 (+ 2760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:15.088821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:15.088824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:15.089006 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:15.089642 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:15.089651 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:15.089656 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:15.090109 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:15.090250 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:15.091815 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0810 19:34:15.092106 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29721,"thread_start_us":29911,"threads_started":1}
I20260810 19:34:15.095423 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:15.039482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:44091 (local address 127.29.233.129:54001)
0810 19:34:15.064218 (+ 24736us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:15.064292 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:15.069364 (+ 5072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:15.070498 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:15.070516 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:15.071488 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:34:15.072396 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:15.072425 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.073566 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.073571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:15.075119 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:15.075137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.076569 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.076580 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:15.076887 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:15.077585 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:34:15.077632 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:34:15.081950 (+ 4318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:34:15.088036 (+ 6086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:34:15.088046 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:34:15.088065 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:34:15.088682 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:34:15.089152 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:34:15.089159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:34:15.089166 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:34:15.089472 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:34:15.090376 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:34:15.090387 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:34:15.091319 (+ 932us) client_negotiation.cc:769] Sending connection context
0810 19:34:15.093388 (+ 2069us) client_negotiation.cc:240] Negotiation successful
0810 19:34:15.093954 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24279,"thread_start_us":26694,"threads_started":1}
I20260810 19:34:15.097354 10930 heartbeater.cc:347] Connected to a master server at 127.29.233.190:44091
I20260810 19:34:15.097694 10930 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
I20260810 19:34:15.098577 10930 heartbeater.cc:511] Master 127.29.233.190:44091 requested a full tablet report, sending...
I20260810 19:34:15.101214 10692 ts_manager.cc:194] Registered new tserver with Master: 38f61de586ac46a9be16d2ae149c3698 (127.29.233.129:33707)
I20260810 19:34:15.103102 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:54001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
W20260810 19:34:15.161042 30630 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.233.130: real 0.120s user 0.001s sys 0.026s
I20260810 19:34:15.162070 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:46751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44091
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:34:15 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390454, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
W20260810 19:34:15.688875 10936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:34:15.689458 10936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:34:15.689677 10936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:34:15.701771 10936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:34:15.702013 10936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:34:15.702271 10936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:34:15.718369 10936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.233.148:46751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:34:15.720569 10936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:34:15.722527 10936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:34:15.773514 10946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:34:15.779915 10945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:34:15.780484 10949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:34:15.780797 10936 server_base.cc:1034] running on GCE node
I20260810 19:34:15.782454 10936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:34:15.790551 10936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:34:15.791920 10936 hybrid_clock.cc:648] HybridClock initialized: now 1786390455791808 us; error 102 us; skew 500 ppm
I20260810 19:34:15.792377 10936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:34:15 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390455, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:34:15.796979 10936 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:34:15.799018 10936 webserver.cc:460] Webserver started at http://127.29.233.130:41919/ using document root <none> and password file <none>
I20260810 19:34:15.799887 10936 fs_manager.cc:359] Metadata directory not provided
I20260810 19:34:15.800032 10936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:34:15.800365 10936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:34:15 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390455, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:34:15.818863 10936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4"
format_stamp: "Formatted at 2026-08-10 19:34:15 on dist-test-slave-tm1g"
server_key: "f0e84540531c61be9e9921467cc396ad"
server_key_iv: "b55499894c880bd9dce8d2c4bf598beb"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:15.820075 10936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4"
format_stamp: "Formatted at 2026-08-10 19:34:15 on dist-test-slave-tm1g"
server_key: "f0e84540531c61be9e9921467cc396ad"
server_key_iv: "b55499894c880bd9dce8d2c4bf598beb"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:15.839201 10936 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.004s sys 0.011s
I20260810 19:34:15.864130 10960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:15.866106 10936 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.000s
I20260810 19:34:15.866351 10936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4"
format_stamp: "Formatted at 2026-08-10 19:34:15 on dist-test-slave-tm1g"
server_key: "f0e84540531c61be9e9921467cc396ad"
server_key_iv: "b55499894c880bd9dce8d2c4bf598beb"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:15.866642 10936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:34:15.895121 10936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:34:15.899580 10936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:34:15.899926 10936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:34:15.901069 10936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:34:15.902704 10936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:34:15.902812 10936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:15.902973 10936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:34:15.903049 10936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:15.948906 10936 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:41959
I20260810 19:34:15.949008 11087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:41959 every 8 connection(s)
I20260810 19:34:15.950281 10936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:34:15.953265 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 10936
I20260810 19:34:15.953454 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:34:15.954023 30630 external_mini_cluster.cc:1428] Setting key dac26f6a79364b94b4b30b6c56e9bc87
Aug 10 19:34:15 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390455, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:34:15.977310 11090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:15.953162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:41109 (local address 127.29.233.190:44091)
0810 19:34:15.959103 (+ 5941us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:15.959109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:15.959129 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:34:15.959196 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:15.959201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:15.959281 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:15.959407 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:15.961600 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.962841 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.965539 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.966179 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.966353 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:15.971000 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:15.971008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:15.971012 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:15.971057 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:15.973933 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:15.974773 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:15.974781 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:15.974785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:15.974931 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:15.975418 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:15.975423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:15.975426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:15.975729 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:15.975906 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:15.976659 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 19:34:15.976918 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5803,"thread_start_us":5657,"threads_started":1}
I20260810 19:34:15.978403 11092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:15.953921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:44091 (local address 127.29.233.130:41109)
0810 19:34:15.955896 (+ 1975us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:15.955941 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:15.958278 (+ 2337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:15.959731 (+ 1453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:15.959747 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:15.960516 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:34:15.961375 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:15.961406 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.963027 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.963035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:15.965359 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:15.965379 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:15.966463 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:15.966473 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:15.966811 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:15.967659 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:34:15.967693 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:34:15.970712 (+ 3019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:34:15.974099 (+ 3387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:34:15.974109 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:34:15.974127 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:34:15.974645 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:34:15.975056 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:34:15.975061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:34:15.975064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:34:15.975292 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:34:15.975866 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:34:15.975874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:34:15.976241 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 19:34:15.976536 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 19:34:15.977093 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1665,"thread_start_us":5470,"threads_started":1}
I20260810 19:34:15.980341 11088 heartbeater.cc:347] Connected to a master server at 127.29.233.190:44091
I20260810 19:34:15.980687 11088 heartbeater.cc:464] Registering TS with master...
I20260810 19:34:15.981442 11088 heartbeater.cc:511] Master 127.29.233.190:44091 requested a full tablet report, sending...
I20260810 19:34:15.983083 10692 ts_manager.cc:194] Registered new tserver with Master: 0c30ba0285cc4c31a8b01b6cd740a5f4 (127.29.233.130:41959)
I20260810 19:34:15.983911 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:41109
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:34:16.026670 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:46751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44091
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:34:16.106907 10930 heartbeater.cc:503] Master 127.29.233.190:44091 was elected leader, sending a full tablet report...
W20260810 19:34:16.382875 11095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:34:16.383214 11095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:34:16.383384 11095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:34:16.392604 11095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:34:16.392747 11095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:34:16.392923 11095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:34:16.405619 11095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.233.148:46751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:44091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:34:16.407169 11095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:34:16.408545 11095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:34:16.422390 11102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:34:16.422461 11101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:34:16.422425 11104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:34:16.423674 11095 server_base.cc:1034] running on GCE node
I20260810 19:34:16.424289 11095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:34:16.425267 11095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:34:16.426554 11095 hybrid_clock.cc:648] HybridClock initialized: now 1786390456426506 us; error 39 us; skew 500 ppm
I20260810 19:34:16.426904 11095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:34:16 dist-test-slave-tm1g krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390456, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:34:16.430135 11095 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:34:16.431751 11095 webserver.cc:460] Webserver started at http://127.29.233.131:46251/ using document root <none> and password file <none>
I20260810 19:34:16.432483 11095 fs_manager.cc:359] Metadata directory not provided
I20260810 19:34:16.432642 11095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:34:16.432964 11095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:34:16 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390456, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:34:16.451498 11095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "2c0ce8149ab042c0a138675dec3969d9"
format_stamp: "Formatted at 2026-08-10 19:34:16 on dist-test-slave-tm1g"
server_key: "32a3cf8aa6d02b61f12c54e4ffab07e4"
server_key_iv: "3ea5011a95bc5bb0ef107beb5b5ac582"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:16.452734 11095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "2c0ce8149ab042c0a138675dec3969d9"
format_stamp: "Formatted at 2026-08-10 19:34:16 on dist-test-slave-tm1g"
server_key: "32a3cf8aa6d02b61f12c54e4ffab07e4"
server_key_iv: "3ea5011a95bc5bb0ef107beb5b5ac582"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:16.459926 11095 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260810 19:34:16.477631 11111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:16.479476 11095 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260810 19:34:16.479629 11095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "2c0ce8149ab042c0a138675dec3969d9"
format_stamp: "Formatted at 2026-08-10 19:34:16 on dist-test-slave-tm1g"
server_key: "32a3cf8aa6d02b61f12c54e4ffab07e4"
server_key_iv: "3ea5011a95bc5bb0ef107beb5b5ac582"
server_key_version: "kuduclusterkey@0"
I20260810 19:34:16.479823 11095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:34:16.495693 11095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:34:16.499571 11095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:34:16.499830 11095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:34:16.500731 11095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:34:16.502065 11095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:34:16.502141 11095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:16.502225 11095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:34:16.502274 11095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:34:16.538395 11095 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:39713
I20260810 19:34:16.538496 11234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:39713 every 8 connection(s)
I20260810 19:34:16.539805 11095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:34:16.548732 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 11095
I20260810 19:34:16.548915 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:34:16.549386 30630 external_mini_cluster.cc:1428] Setting key 1889e5a08cfa014bdb067eced5812dce
Aug 10 19:34:16 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390456, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:34:16.560555 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.542508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:47421 (local address 127.29.233.190:44091)
0810 19:34:16.542969 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.542974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.544005 (+ 1031us) server_negotiation.cc:407] Connection header received
0810 19:34:16.545816 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.545822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.545938 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.546059 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:16.547558 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.548305 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.550353 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.551025 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.551157 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.554895 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:16.554900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:16.554904 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:16.554943 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:16.557645 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:16.558423 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:16.558428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:16.558430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:16.558568 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:16.558984 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:16.558988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:16.558990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:16.559264 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:16.559424 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.560083 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.560265 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":218,"threads_started":1}
I20260810 19:34:16.561503 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.543119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:44091 (local address 127.29.233.131:47421)
0810 19:34:16.543833 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:16.543880 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:16.545543 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:16.546342 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:16.546355 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:16.546801 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:34:16.547379 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.547397 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.548500 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.548504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.550192 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.550206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.551257 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.551262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.551598 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.552338 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:34:16.552361 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:34:16.554667 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:34:16.557772 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:34:16.557778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:34:16.557791 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:34:16.558237 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:34:16.558676 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:34:16.558679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:34:16.558681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:34:16.558850 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:34:16.559423 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:34:16.559430 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:34:16.559730 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 19:34:16.559970 (+ 240us) client_negotiation.cc:240] Negotiation successful
0810 19:34:16.560341 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":309,"threads_started":1}
I20260810 19:34:16.562980 11235 heartbeater.cc:347] Connected to a master server at 127.29.233.190:44091
I20260810 19:34:16.563350 11235 heartbeater.cc:464] Registering TS with master...
I20260810 19:34:16.564338 11235 heartbeater.cc:511] Master 127.29.233.190:44091 requested a full tablet report, sending...
I20260810 19:34:16.565922 10692 ts_manager.cc:194] Registered new tserver with Master: 2c0ce8149ab042c0a138675dec3969d9 (127.29.233.131:39713)
I20260810 19:34:16.566820 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:47421
I20260810 19:34:16.577615 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:34:16.595983 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.582545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.29.233.190:44091)
0810 19:34:16.582748 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.582757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.583242 (+ 485us) server_negotiation.cc:407] Connection header received
0810 19:34:16.583459 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.583464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.583551 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.583656 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:16.585224 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.585990 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.587399 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.587912 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.588046 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.590632 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:16.590639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:16.590650 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:16.590682 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:16.593240 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:16.593924 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:16.593929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:16.593933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:16.594073 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:16.594546 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:16.594551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:16.594554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:16.594828 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:16.595088 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.595405 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.595652 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 19:34:16.600046 10692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:34:16.607486 10692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:34:16.654173 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.621220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:39713 (local address 127.0.0.1:52624)
0810 19:34:16.640013 (+ 18793us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:16.640028 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:16.640170 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:16.641235 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:16.641243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:16.641273 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:34:16.641798 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.641812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.644656 (+ 2844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.644662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.649823 (+ 5161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.649840 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.653107 (+ 3267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.653114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.653319 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.653382 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 19:34:16.653528 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 19:34:16.653736 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18675,"mutex_wait_us":1,"thread_start_us":18516,"threads_started":1}
I20260810 19:34:16.655109 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.620995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.29.233.130:41959)
0810 19:34:16.641997 (+ 21002us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.642006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.642036 (+ 30us) server_negotiation.cc:407] Connection header received
0810 19:34:16.642127 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.642137 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.642401 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.642604 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:34:16.645698 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.647527 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.650233 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.653251 (+ 3018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.654345 (+ 1094us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.654533 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.654610 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.654785 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20813,"thread_start_us":21767,"threads_started":1}
I20260810 19:34:16.654618 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.620784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:41959 (local address 127.0.0.1:33930)
0810 19:34:16.637129 (+ 16345us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:16.637199 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:16.637398 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:16.642607 (+ 5209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:16.642612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:16.642636 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:34:16.643027 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.643046 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.647731 (+ 4685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.647739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.650039 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.650054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.653543 (+ 3489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.653553 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.653816 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.653876 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 19:34:16.653955 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 19:34:16.654126 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15964,"thread_start_us":15755,"threads_started":1}
I20260810 19:34:16.656174 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.621312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.29.233.131:39713)
0810 19:34:16.640651 (+ 19339us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.640658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.640686 (+ 28us) server_negotiation.cc:407] Connection header received
0810 19:34:16.640765 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.640770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.641008 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.641238 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:34:16.642063 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.644488 (+ 2425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.650102 (+ 5614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.652525 (+ 2423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.652720 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.652918 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.655663 (+ 2745us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.655836 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19188,"thread_start_us":22830,"threads_started":1}
I20260810 19:34:16.659755 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.637740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:33707 (local address 127.0.0.1:42638)
0810 19:34:16.650696 (+ 12956us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:16.650715 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:16.650889 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:16.651838 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:16.651844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:16.651885 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:34:16.652326 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.652338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.654438 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.654442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.656823 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.656837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.658983 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.658992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.659165 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.659208 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 19:34:16.659305 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 19:34:16.659445 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12828,"thread_start_us":12723,"threads_started":1}
I20260810 19:34:16.660519 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.638295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.29.233.129:33707)
0810 19:34:16.649954 (+ 11659us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.649967 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.650894 (+ 927us) server_negotiation.cc:407] Connection header received
0810 19:34:16.651051 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.651060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.651325 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.651572 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:34:16.652545 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.654225 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.657004 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.658401 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.658647 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.658823 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.660026 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.660203 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11386,"thread_start_us":11650,"threads_started":1}
I20260810 19:34:16.660794 11162 tablet_service.cc:1467] Processing CreateTablet for tablet 58988e7a496f4168a44211258524361e (DEFAULT_TABLE table=test-table [id=4081b9f03f08426d9fc4cb24f65f3f8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:34:16.662844 11016 tablet_service.cc:1467] Processing CreateTablet for tablet 58988e7a496f4168a44211258524361e (DEFAULT_TABLE table=test-table [id=4081b9f03f08426d9fc4cb24f65f3f8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:34:16.664459 11016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58988e7a496f4168a44211258524361e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:34:16.664515 10858 tablet_service.cc:1467] Processing CreateTablet for tablet 58988e7a496f4168a44211258524361e (DEFAULT_TABLE table=test-table [id=4081b9f03f08426d9fc4cb24f65f3f8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:34:16.665925 10858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58988e7a496f4168a44211258524361e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:34:16.665923 11162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58988e7a496f4168a44211258524361e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:34:16.680601 11255 tablet_bootstrap.cc:492] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Bootstrap starting.
I20260810 19:34:16.684564 11255 tablet_bootstrap.cc:654] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Neither blocks nor log segments found. Creating new log.
I20260810 19:34:16.685717 11255 log.cc:824] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Log is configured to *not* fsync() on all Append() calls
I20260810 19:34:16.687101 11256 tablet_bootstrap.cc:492] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Bootstrap starting.
I20260810 19:34:16.688819 11255 tablet_bootstrap.cc:492] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: No bootstrap required, opened a new log
I20260810 19:34:16.689170 11255 ts_tablet_manager.cc:1397] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 19:34:16.692092 11257 tablet_bootstrap.cc:492] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Bootstrap starting.
I20260810 19:34:16.694262 11256 tablet_bootstrap.cc:654] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Neither blocks nor log segments found. Creating new log.
I20260810 19:34:16.695703 11256 log.cc:824] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Log is configured to *not* fsync() on all Append() calls
I20260810 19:34:16.695901 11257 tablet_bootstrap.cc:654] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Neither blocks nor log segments found. Creating new log.
I20260810 19:34:16.697240 11257 log.cc:824] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Log is configured to *not* fsync() on all Append() calls
I20260810 19:34:16.698987 11255 raft_consensus.cc:348] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.699451 11255 raft_consensus.cc:374] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:34:16.699576 11255 raft_consensus.cc:729] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c30ba0285cc4c31a8b01b6cd740a5f4, State: Initialized, Role: FOLLOWER
I20260810 19:34:16.700248 11255 consensus_queue.cc:260] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.700704 11256 tablet_bootstrap.cc:492] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: No bootstrap required, opened a new log
I20260810 19:34:16.701078 11256 ts_tablet_manager.cc:1397] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260810 19:34:16.701321 11257 tablet_bootstrap.cc:492] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: No bootstrap required, opened a new log
I20260810 19:34:16.701678 11257 ts_tablet_manager.cc:1397] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 19:34:16.711102 11088 heartbeater.cc:503] Master 127.29.233.190:44091 was elected leader, sending a full tablet report...
I20260810 19:34:16.711918 11255 ts_tablet_manager.cc:1428] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Time spent starting tablet: real 0.023s user 0.015s sys 0.007s
I20260810 19:34:16.714882 11257 raft_consensus.cc:348] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.715446 11257 raft_consensus.cc:374] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:34:16.715590 11257 raft_consensus.cc:729] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0ce8149ab042c0a138675dec3969d9, State: Initialized, Role: FOLLOWER
I20260810 19:34:16.716372 11257 consensus_queue.cc:260] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.716389 11256 raft_consensus.cc:348] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.716842 11256 raft_consensus.cc:374] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:34:16.716953 11256 raft_consensus.cc:729] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38f61de586ac46a9be16d2ae149c3698, State: Initialized, Role: FOLLOWER
I20260810 19:34:16.717639 11235 heartbeater.cc:503] Master 127.29.233.190:44091 was elected leader, sending a full tablet report...
I20260810 19:34:16.717578 11256 consensus_queue.cc:260] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.718677 11257 ts_tablet_manager.cc:1428] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Time spent starting tablet: real 0.017s user 0.008s sys 0.006s
I20260810 19:34:16.719354 11256 ts_tablet_manager.cc:1428] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
W20260810 19:34:16.791543 11236 tablet.cc:2369] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:34:16.820384 11261 raft_consensus.cc:482] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:34:16.820693 11261 raft_consensus.cc:504] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.822520 11261 leader_election.cc:288] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0ce8149ab042c0a138675dec3969d9 (127.29.233.131:39713), 38f61de586ac46a9be16d2ae149c3698 (127.29.233.129:33707)
W20260810 19:34:16.823752 10931 tablet.cc:2369] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:34:16.829793 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.822937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:37805 (local address 127.29.233.131:39713)
0810 19:34:16.823377 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.823383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.823730 (+ 347us) server_negotiation.cc:407] Connection header received
0810 19:34:16.824094 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.824100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.824186 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.824266 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:34:16.825179 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.825938 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.827929 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.828742 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.828933 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.829022 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.829394 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.829530 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260810 19:34:16.829864 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.823099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:56273 (local address 127.29.233.129:33707)
0810 19:34:16.823546 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.823555 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.824051 (+ 496us) server_negotiation.cc:407] Connection header received
0810 19:34:16.824325 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.824331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.824409 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.824530 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:34:16.825331 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.826061 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.827924 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.828725 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.828932 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.829016 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.829457 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.829591 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260810 19:34:16.830106 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.822657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:39713 (local address 127.29.233.130:37805)
0810 19:34:16.823610 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:16.823633 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:16.823869 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:16.824487 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:16.824493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:16.824523 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:34:16.825015 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.825027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.826080 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.826086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.827778 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.827790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.829033 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.829041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.829206 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.829264 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 19:34:16.829369 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 19:34:16.829504 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":107,"thread_start_us":561,"threads_started":1}
I20260810 19:34:16.830189 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.823007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:33707 (local address 127.29.233.130:56273)
0810 19:34:16.823941 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0810 19:34:16.823961 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:34:16.824162 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:34:16.824584 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:34:16.824589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:34:16.824654 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:34:16.825155 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.825166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.826206 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.826217 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.827763 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:34:16.827775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.828980 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.828987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:34:16.829203 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.829256 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 19:34:16.829429 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 19:34:16.829561 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":57,"spinlock_wait_cycles":1408,"thread_start_us":557,"threads_started":1}
I20260810 19:34:16.831331 11185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58988e7a496f4168a44211258524361e" candidate_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0ce8149ab042c0a138675dec3969d9" is_pre_election: true
I20260810 19:34:16.831331 10879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58988e7a496f4168a44211258524361e" candidate_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38f61de586ac46a9be16d2ae149c3698" is_pre_election: true
I20260810 19:34:16.831952 10879 raft_consensus.cc:2454] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c30ba0285cc4c31a8b01b6cd740a5f4 in term 0.
I20260810 19:34:16.831962 11185 raft_consensus.cc:2454] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c30ba0285cc4c31a8b01b6cd740a5f4 in term 0.
I20260810 19:34:16.832698 10968 leader_election.cc:302] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c30ba0285cc4c31a8b01b6cd740a5f4, 2c0ce8149ab042c0a138675dec3969d9, 38f61de586ac46a9be16d2ae149c3698; no voters:
I20260810 19:34:16.833197 11261 raft_consensus.cc:2781] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:34:16.833370 11261 raft_consensus.cc:482] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:34:16.833510 11261 raft_consensus.cc:3037] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:34:16.836180 11261 raft_consensus.cc:504] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.836851 11261 leader_election.cc:288] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0ce8149ab042c0a138675dec3969d9 (127.29.233.131:39713), 38f61de586ac46a9be16d2ae149c3698 (127.29.233.129:33707)
I20260810 19:34:16.837330 10879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58988e7a496f4168a44211258524361e" candidate_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38f61de586ac46a9be16d2ae149c3698"
I20260810 19:34:16.837374 11185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58988e7a496f4168a44211258524361e" candidate_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0ce8149ab042c0a138675dec3969d9"
I20260810 19:34:16.837544 10879 raft_consensus.cc:3037] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:34:16.837581 11185 raft_consensus.cc:3037] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:34:16.840036 10879 raft_consensus.cc:2454] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c30ba0285cc4c31a8b01b6cd740a5f4 in term 1.
I20260810 19:34:16.840477 11185 raft_consensus.cc:2454] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c30ba0285cc4c31a8b01b6cd740a5f4 in term 1.
I20260810 19:34:16.840504 10969 leader_election.cc:302] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c30ba0285cc4c31a8b01b6cd740a5f4, 38f61de586ac46a9be16d2ae149c3698; no voters:
I20260810 19:34:16.840842 11261 raft_consensus.cc:2781] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:34:16.841328 11261 raft_consensus.cc:686] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 1 LEADER]: Becoming Leader. State: Replica: 0c30ba0285cc4c31a8b01b6cd740a5f4, State: Running, Role: LEADER
I20260810 19:34:16.841912 11261 consensus_queue.cc:237] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } }
I20260810 19:34:16.846647 10691 catalog_manager.cc:5495] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c30ba0285cc4c31a8b01b6cd740a5f4 (127.29.233.130). New cstate: current_term: 1 leader_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c30ba0285cc4c31a8b01b6cd740a5f4" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 } health_report { overall_health: UNKNOWN } } }
I20260810 19:34:16.878353 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.872698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.29.233.130:41959)
0810 19:34:16.872899 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.872904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.872927 (+ 23us) server_negotiation.cc:407] Connection header received
0810 19:34:16.873112 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.873116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.873194 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.873318 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:34:16.873771 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.874579 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.876133 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.876524 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.876618 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.877135 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:34:16.877678 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:34:16.877792 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.877904 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.877976 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:34:16.887131 10879 raft_consensus.cc:1261] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 1 FOLLOWER]: Refusing update from remote peer 0c30ba0285cc4c31a8b01b6cd740a5f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:34:16.887128 11185 raft_consensus.cc:1261] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 1 FOLLOWER]: Refusing update from remote peer 0c30ba0285cc4c31a8b01b6cd740a5f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:34:16.888269 11261 consensus_queue.cc:1035] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38f61de586ac46a9be16d2ae149c3698" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:34:16.888617 11268 consensus_queue.cc:1035] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0ce8149ab042c0a138675dec3969d9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:34:16.900908 11271 mvcc.cc:205] Tried to move back new op lower bound from 7317055311398699008 to 7317055311228952576. Current Snapshot: MvccSnapshot[applied={T|T < 7317055311398699008}]
I20260810 19:34:16.907586 11273 mvcc.cc:205] Tried to move back new op lower bound from 7317055311398699008 to 7317055311228952576. Current Snapshot: MvccSnapshot[applied={T|T < 7317055311398699008}]
I20260810 19:34:16.913998 10691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:34:16.914290 10691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:34:16.917999 10691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fa70e6de98846febdcb2a9842696070: Sending DeleteTablet for 3 replicas of tablet 58988e7a496f4168a44211258524361e
I20260810 19:34:16.919322 11016 tablet_service.cc:1514] Processing DeleteTablet for tablet 58988e7a496f4168a44211258524361e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:34:16 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:33930
I20260810 19:34:16.919420 11162 tablet_service.cc:1514] Processing DeleteTablet for tablet 58988e7a496f4168a44211258524361e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:34:16 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52624
I20260810 19:34:16.919723 10858 tablet_service.cc:1514] Processing DeleteTablet for tablet 58988e7a496f4168a44211258524361e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:34:16 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:42638
I20260810 19:34:16.920318 11281 tablet_replica.cc:330] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: stopping tablet replica
I20260810 19:34:16.920846 11281 raft_consensus.cc:2229] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:34:16.921469 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 10782
I20260810 19:34:16.921582 11281 raft_consensus.cc:2258] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:34:16.926723 11281 ts_tablet_manager.cc:1905] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:34:16.935523 11283 tablet_replica.cc:330] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: stopping tablet replica
I20260810 19:34:16.936131 11283 raft_consensus.cc:2229] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:34:16.936625 11281 ts_tablet_manager.cc:1918] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:34:16.936789 11283 raft_consensus.cc:2258] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:34:16.936918 11281 log.cc:1196] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/58988e7a496f4168a44211258524361e
I20260810 19:34:16.937541 11281 ts_tablet_manager.cc:1939] T 58988e7a496f4168a44211258524361e P 0c30ba0285cc4c31a8b01b6cd740a5f4: Deleting consensus metadata
I20260810 19:34:16.939393 11282 tablet_replica.cc:330] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: stopping tablet replica
I20260810 19:34:16.939730 10677 catalog_manager.cc:4841] TS 0c30ba0285cc4c31a8b01b6cd740a5f4 (127.29.233.130:41959): tablet 58988e7a496f4168a44211258524361e (table test-table [id=4081b9f03f08426d9fc4cb24f65f3f8e]) successfully deleted
I20260810 19:34:16.939802 11282 raft_consensus.cc:2229] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:34:16.940332 11282 raft_consensus.cc:2258] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:34:16.941009 11283 ts_tablet_manager.cc:1905] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:34:16.942559 11282 ts_tablet_manager.cc:1905] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 19:34:16 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:34:16.950119 11282 ts_tablet_manager.cc:1918] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:34:16.950393 11282 log.cc:1196] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/58988e7a496f4168a44211258524361e
I20260810 19:34:16.951027 11282 ts_tablet_manager.cc:1939] T 58988e7a496f4168a44211258524361e P 2c0ce8149ab042c0a138675dec3969d9: Deleting consensus metadata
I20260810 19:34:16.952998 10675 catalog_manager.cc:4841] TS 2c0ce8149ab042c0a138675dec3969d9 (127.29.233.131:39713): tablet 58988e7a496f4168a44211258524361e (table test-table [id=4081b9f03f08426d9fc4cb24f65f3f8e]) successfully deleted
I20260810 19:34:16.953992 11283 ts_tablet_manager.cc:1918] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:34:16.954171 11283 log.cc:1196] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/58988e7a496f4168a44211258524361e
I20260810 19:34:16.954777 11283 ts_tablet_manager.cc:1939] T 58988e7a496f4168a44211258524361e P 38f61de586ac46a9be16d2ae149c3698: Deleting consensus metadata
I20260810 19:34:16.956599 10677 catalog_manager.cc:4841] TS 38f61de586ac46a9be16d2ae149c3698 (127.29.233.129:33707): tablet 58988e7a496f4168a44211258524361e (table test-table [id=4081b9f03f08426d9fc4cb24f65f3f8e]) successfully deleted
I20260810 19:34:16.956944 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:16.922679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.29.233.129:33707)
0810 19:34:16.922863 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:16.922870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:16.934868 (+ 11998us) server_negotiation.cc:407] Connection header received
0810 19:34:16.935407 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:16.935413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:16.935497 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:16.935581 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:16.937856 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.939097 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.941866 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:16.942594 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:16.942708 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:16.948530 (+ 5822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:16.948546 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:16.948562 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:16.948611 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:16.952620 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:16.953641 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:16.953653 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:16.953659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:16.953855 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:16.954563 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:16.954568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:16.954571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:16.955160 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:16.955510 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:16.956185 (+ 675us) server_negotiation.cc:299] Negotiation successful
0810 19:34:16.956461 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 19:34:16.958215 10924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:34:17.051512 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 10782
I20260810 19:34:17.073012 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 10936
Aug 10 19:34:17 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:34:17.090106 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:17.073840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.29.233.130:41959)
0810 19:34:17.074022 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:17.074029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:17.074079 (+ 50us) server_negotiation.cc:407] Connection header received
0810 19:34:17.074338 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:17.074345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:17.074431 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:17.074538 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:17.075883 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:17.076741 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:17.078045 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:17.078676 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:17.078815 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:17.083495 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:17.083509 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:17.083526 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:17.083563 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:17.087249 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:17.088125 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:17.088132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:17.088138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:17.088270 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:17.088676 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:17.088679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:17.088681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:17.089023 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:17.089252 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:17.089631 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 19:34:17.089799 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:34:17.090893 11083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:34:17.207497 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 10936
I20260810 19:34:17.229633 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 11095
Aug 10 19:34:17 dist-test-slave-tm1g krb5kdc[10161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:34:17.245370 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:34:17.230482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.29.233.131:39713)
0810 19:34:17.230655 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 19:34:17.230660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:34:17.230722 (+ 62us) server_negotiation.cc:407] Connection header received
0810 19:34:17.230874 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:34:17.230878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:34:17.230944 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:34:17.231030 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:34:17.232427 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:17.233312 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:17.234847 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:34:17.235233 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:34:17.235328 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:34:17.239813 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:34:17.239827 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:34:17.239843 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:34:17.239880 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:34:17.242562 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:17.243285 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:17.243302 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:17.243304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:17.243452 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:34:17.243895 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:34:17.243899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:34:17.243901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:34:17.244297 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:34:17.244558 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 19:34:17.244882 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 19:34:17.245059 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:34:17.246027 11230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:34:17.337357 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 11095
I20260810 19:34:17.357118 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 10654
I20260810 19:34:17.358481 10726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:34:17.474030 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 10654
I20260810 19:34:17.505939 30630 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 19:34:17.851791 30630 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 19:34:17.852149 30630 mini_ranger.cc:66] Stopping Ranger...
I20260810 19:34:18.197721 30630 mini_ranger.cc:68] Stopped Ranger
2026-08-10 19:34:18.198 UTC [10191] LOG: received smart shutdown request
2026-08-10 19:34:18.202 UTC [10191] LOG: background worker "logical replication launcher" (PID 10203) exited with exit code 1
2026-08-10 19:34:18.202 UTC [10198] LOG: shutting down
2026-08-10 19:34:18.387 UTC [10191] LOG: database system is shut down
2026-08-10T19:34:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51011 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:34:18 dist-test-slave-tm1g krb5kdc[11299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:34:18 dist-test-slave-tm1g krb5kdc[11299](info): set up 2 sockets
Aug 10 19:34:18 dist-test-slave-tm1g krb5kdc[11299](info): commencing operation
krb5kdc: starting...
W20260810 19:34:19.711021 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.007s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:34:19 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:34:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:34:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/rangeradmin_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/rangeradmin_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/rangerlookup_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/rangerlookup_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
I20260810 19:34:20.104655 30630 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskW7W1IE/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/postgres -l logfile start
2026-08-10 19:34:22.663 UTC [11333] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 19:34:22.663 UTC [11333] LOG: listening on IPv4 address "127.29.233.148", port 56107
2026-08-10 19:34:22.668 UTC [11333] LOG: listening on Unix socket "/tmp/.s.PGSQL.56107"
2026-08-10 19:34:22.692 UTC [11336] LOG: database system was shut down at 2026-08-10 19:34:21 UTC
2026-08-10 19:34:22.696 UTC [11333] LOG: database system is ready to accept connections
I20260810 19:34:23.725935 30630 mini_postgres.cc:96] Postgres bound to 56107
2026-08-10 19:34:23.753 UTC [11345] FATAL: database "slave" does not exist
127.29.233.148:56107 - accepting connections
I20260810 19:34:23.754655 30630 mini_ranger.cc:161] Starting Ranger...
I20260810 19:34:23.788100 30630 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 19:34:24.298496 30630 mini_ranger.cc:87] Created ranger Postgres database
I20260810 19:34:24.298604 30630 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-admin
2026-08-10 19:34:24,603 [I] DB FLAVOR :POSTGRES
2026-08-10 19:34:24,604 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 19:34:24,604 [I] Checking connection..
2026-08-10 19:34:24,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 19:34:24,881 [I] Checking connection passed.
2026-08-10 19:34:24,882 [I] --------- Verifying version history table ---------
2026-08-10 19:34:24,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 19:34:25,132 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 19:34:25,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 19:34:25,397 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 19:34:25,398 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 19:34:25,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 19:34:25.645 UTC [11416] WARNING: there is no transaction in progress
2026-08-10 19:34:25,652 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 19:34:25,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 19:34:25,908 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 19:34:25,909 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 19:34:25,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 19:34:26,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 19:34:26,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-10 19:34:26,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 19:34:26,951 [I] Table x_portal_user does not exist in database ranger
2026-08-10 19:34:26,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 19:34:27,221 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 19:34:27,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 19:34:27,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 19:34:27,725 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 19:34:27,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 19:34:28.733 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.751 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.769 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.786 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.802 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.822 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.880 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.888 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.903 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.913 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.923 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.938 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.949 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.956 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.963 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:28.971 UTC [11535] WARNING: there is no transaction in progress
2026-08-10 19:34:29,662 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 19:34:29,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 19:34:29,912 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 19:34:29,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 19:34:30,158 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 19:34:30,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 19:34:30,427 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 19:34:30,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 19:34:30,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 19:34:30,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-10 19:34:31,202 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 19:34:31,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 19:34:31,468 [I] DB_PATCHES have already been applied
I20260810 19:34:31.474594 30630 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-admin:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskW7W1IE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 19:34:31.474685 30630 mini_ranger.cc:193] Using host: 127.29.233.148
I20260810 19:34:31.496785 30630 mini_ranger.cc:227] Ranger admin URL: http://127.29.233.148:49869
Aug 10, 2026 7:34:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:34:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:34:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:34:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:34:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:34:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:34:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 19:34:32.722352 30630 mini_ranger.cc:160] Time spent starting Ranger: real 8.968s user 0.002s sys 0.062s
Aug 10, 2026 7:34:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:34:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 7:34:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:34:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:34:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 19:34:53.613651 30630 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.233.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.233.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/rangerkms_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.233.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/rangerkms_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.233.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.233.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.233.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/HTTP_127.29.233.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 19:34:53 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390493, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 19:34:53.839196 30630 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 19:34:53.868232 30630 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 19:34:54.797350 30630 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 19:34:54.797500 30630 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms
I20260810 19:34:54.797571 30630 mini_ranger_kms.cc:227] Using postgres at 127.29.233.148:56107
2026-08-10 19:34:55,165 [I] DB FLAVOR :POSTGRES
2026-08-10 19:34:55,166 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 19:34:55,166 [I] Checking connection
2026-08-10 19:34:55,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 19:34:55,426 [I] connection success
2026-08-10 19:34:55,426 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 19:34:55,426 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 19:34:55,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 19:34:55,675 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 19:34:55,675 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 19:34:55,675 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 19:34:55,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.233.148:56107/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 19:34:55,959 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 19:34:56.411270 30630 mini_ranger_kms.cc:310] Created kms service
I20260810 19:34:56.632490 30630 mini_ranger_kms.cc:326] Created kudu user
I20260810 19:34:56.680225 30630 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 19:34:56 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390493, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:34:57.126811 30630 mini_ranger_kms.cc:384] Added ranger policy
I20260810 19:34:57.127180 30630 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms:/tmp/dist-test-taskW7W1IE/build/asan/bin/postgresql.jar:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskW7W1IE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 19:34:57.127238 30630 mini_ranger_kms.cc:240] Using host: 127.29.233.148
I20260810 19:34:57.154372 30630 mini_ranger_kms.cc:276] Ranger KMS PID: 11739
I20260810 19:34:57.154512 30630 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.233.148:36999
Aug 10, 2026 7:34:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:34:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:34:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:34:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:34:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:34:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 19:34:59 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786390499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.233.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:34:59.611588 30630 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.772s user 0.093s sys 0.313s
I20260810 19:34:59.611861 30630 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 19:34:59.612056 30630 mini_ranger_kms.cc:401] 127.29.233.148:36999/kms/v1/keys
Aug 10 19:34:59 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.233.148@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
Aug 10 19:35:00 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.233.148@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
Aug 10 19:35:02 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:35:02.708405 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:36315
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:36315
--ranger_config_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:03.058892 11796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:03.059206 11796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:03.059360 11796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:03.066735 11796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:03.066848 11796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:03.066939 11796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:03.067019 11796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:03.067121 11796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:03.077358 11796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.233.148:36999/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:36315
--ranger_config_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:36315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:03.078739 11796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:03.080107 11796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:03.092677 11803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:03.092768 11806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:03.092813 11804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:03.093305 11796 server_base.cc:1034] running on GCE node
I20260810 19:35:03.094152 11796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:03.095619 11796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:03.096875 11796 hybrid_clock.cc:648] HybridClock initialized: now 1786390503096828 us; error 58 us; skew 500 ppm
I20260810 19:35:03.097277 11796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:03 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390503, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:03.106017 11796 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:03.107833 11796 webserver.cc:460] Webserver started at http://127.29.233.190:42313/ using document root <none> and password file <none>
I20260810 19:35:03.108552 11796 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:03.108693 11796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:03.109009 11796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:35:03 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390503, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:03.161080 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "c6dab78496e24ad08195e57d347c166f"
format_stamp: "Formatted at 2026-08-10 19:35:03 on dist-test-slave-tm1g"
server_key: "1285c45908d097356bbe61d783b40713"
server_key_iv: "345fff8d8479b8591cac0d470b8e6986"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:03.161978 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "c6dab78496e24ad08195e57d347c166f"
format_stamp: "Formatted at 2026-08-10 19:35:03 on dist-test-slave-tm1g"
server_key: "1285c45908d097356bbe61d783b40713"
server_key_iv: "345fff8d8479b8591cac0d470b8e6986"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:03.167749 11796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:35:03.188189 11813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:03.189743 11796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:35:03.189944 11796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "c6dab78496e24ad08195e57d347c166f"
format_stamp: "Formatted at 2026-08-10 19:35:03 on dist-test-slave-tm1g"
server_key: "1285c45908d097356bbe61d783b40713"
server_key_iv: "345fff8d8479b8591cac0d470b8e6986"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:03.190184 11796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:03.231623 11796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:03.235828 11796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:03.236171 11796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:03.257300 11796 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:36315
I20260810 19:35:03.257304 11870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:36315 every 8 connection(s)
I20260810 19:35:03.258831 11796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:03.262108 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 11796
I20260810 19:35:03.262364 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:35:03.262939 11871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:03.263281 30630 external_mini_cluster.cc:1428] Setting key 38afee7322fabd1f41944bfda99e2d39
I20260810 19:35:03.281634 11871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Bootstrap starting.
I20260810 19:35:03.285609 11871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:03.287128 11871 log.cc:824] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:03.292496 11871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: No bootstrap required, opened a new log
Aug 10 19:35:03 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:03.302264 11871 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:03.302799 11871 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:03.302937 11871 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6dab78496e24ad08195e57d347c166f, State: Initialized, Role: FOLLOWER
I20260810 19:35:03.303810 11871 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:03.304147 11871 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:35:03.304348 11871 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:35:03.304631 11871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:03.305775 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:03.265497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.29.233.190:36315)
0810 19:35:03.279005 (+ 13508us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:03.279022 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:03.279713 (+ 691us) server_negotiation.cc:407] Connection header received
0810 19:35:03.282062 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:03.282097 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:03.282707 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:03.283141 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:03.284924 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:03.287057 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:03.289478 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:03.290598 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:03.290773 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:03.296049 (+ 5276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:03.296060 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:03.296080 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:03.296132 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:03.300538 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:03.301515 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:03.301525 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:03.301542 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:03.301742 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:03.302226 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:03.302231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:03.302234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:03.303084 (+ 850us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:03.303393 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:03.303879 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 19:35:03.304314 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13241,"thread_start_us":13001,"threads_started":1}
I20260810 19:35:03.307376 11871 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:03.308096 11871 leader_election.cc:302] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6dab78496e24ad08195e57d347c166f; no voters:
I20260810 19:35:03.309216 11871 leader_election.cc:288] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:35:03.309314 11877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:03.312537 11871 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:03.316368 11877 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 LEADER]: Becoming Leader. State: Replica: c6dab78496e24ad08195e57d347c166f, State: Running, Role: LEADER
I20260810 19:35:03.317104 11877 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:03.323472 11879 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6dab78496e24ad08195e57d347c166f. Latest consensus state: current_term: 1 leader_uuid: "c6dab78496e24ad08195e57d347c166f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } } }
I20260810 19:35:03.323985 11879 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:03.324644 11878 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6dab78496e24ad08195e57d347c166f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } } }
I20260810 19:35:03.324939 11878 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:03.331421 11886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:03.338181 11886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:03.339095 11871 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 19:35:03.359712 11886 catalog_manager.cc:1347] Generated new cluster ID: 0936e7f38be2477aa040e3109669c771
I20260810 19:35:03.359871 11886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:35:03.375552 11886 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:35:03.378824 11886 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:35:03.390491 11886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Generated new TSK 0
I20260810 19:35:03.391381 11886 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:35:04.219333 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:36315
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:35:04 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786390504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 19:35:04.676081 11921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:04.676407 11921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:04.676586 11921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:04.684120 11921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:04.684262 11921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:04.684459 11921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:35:04.697069 11921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.233.148:36999/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:04.698629 11921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:04.700066 11921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:04.720747 11932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:04.722900 11933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:04.730419 11935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:04.731014 11921 server_base.cc:1034] running on GCE node
I20260810 19:35:04.732190 11921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:04.734933 11921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:04.736274 11921 hybrid_clock.cc:648] HybridClock initialized: now 1786390504736162 us; error 100 us; skew 500 ppm
I20260810 19:35:04.736696 11921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:04 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:04.740518 11921 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:35:04.743752 11921 webserver.cc:460] Webserver started at http://127.29.233.129:35269/ using document root <none> and password file <none>
I20260810 19:35:04.744545 11921 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:04.744693 11921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:04.745071 11921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:35:04 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:04.763301 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "9b694642140a43ef9221282873844de9"
format_stamp: "Formatted at 2026-08-10 19:35:04 on dist-test-slave-tm1g"
server_key: "a07da0f4ce8469f8bb47ca81a8f325ed"
server_key_iv: "17e031e09c3ccef9dc98a7f3ba637e23"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:04.765256 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "9b694642140a43ef9221282873844de9"
format_stamp: "Formatted at 2026-08-10 19:35:04 on dist-test-slave-tm1g"
server_key: "a07da0f4ce8469f8bb47ca81a8f325ed"
server_key_iv: "17e031e09c3ccef9dc98a7f3ba637e23"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:04.786460 11921 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.013s sys 0.004s
I20260810 19:35:04.821542 11942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:04.823750 11921 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.010s sys 0.000s
I20260810 19:35:04.824082 11921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "9b694642140a43ef9221282873844de9"
format_stamp: "Formatted at 2026-08-10 19:35:04 on dist-test-slave-tm1g"
server_key: "a07da0f4ce8469f8bb47ca81a8f325ed"
server_key_iv: "17e031e09c3ccef9dc98a7f3ba637e23"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:04.824435 11921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:04.851504 11921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:04.856101 11921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:04.856439 11921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:04.857499 11921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:04.859032 11921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:04.859110 11921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:04.859259 11921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:04.859326 11921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:04.935837 11921 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:38467
I20260810 19:35:04.935880 12068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:38467 every 8 connection(s)
I20260810 19:35:04.937788 11921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:35:04.948016 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 11921
I20260810 19:35:04.948290 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:35:04.949385 30630 external_mini_cluster.cc:1428] Setting key 8a578adee4ae43d2916de0ab82d90fc7
Aug 10 19:35:04 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:05.006157 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:04.944285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:54521 (local address 127.29.233.190:36315)
0810 19:35:04.978714 (+ 34429us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:04.978722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:04.978753 (+ 31us) server_negotiation.cc:407] Connection header received
0810 19:35:04.984675 (+ 5922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:04.984682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:04.984797 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:04.984895 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:04.988597 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:04.989807 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:04.992302 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:04.992972 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:04.993074 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:04.997878 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:04.997887 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:04.997891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:04.997950 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:05.002764 (+ 4814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:05.003568 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:05.003573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:05.003579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:05.003729 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:05.004222 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:05.004232 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:05.004235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:05.004513 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:05.004674 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:05.005508 (+ 834us) server_negotiation.cc:299] Negotiation successful
0810 19:35:05.005739 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34293,"thread_start_us":38035,"threads_started":1}
I20260810 19:35:05.007784 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:04.941830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:36315 (local address 127.29.233.129:54521)
0810 19:35:04.977539 (+ 35709us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:04.977614 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:04.982018 (+ 4404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:04.985595 (+ 3577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:04.985609 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:04.987427 (+ 1818us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:04.988347 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:04.988376 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:04.990030 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:04.990038 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:04.992116 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:04.992132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:04.993444 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:04.993451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:04.993743 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:04.994605 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:04.994635 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:04.997575 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:05.002935 (+ 5360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:05.002945 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:05.002964 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:05.003426 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:05.003851 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:05.003856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:05.003859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:05.004088 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:05.004650 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:05.004658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:05.005118 (+ 460us) client_negotiation.cc:769] Sending connection context
0810 19:35:05.005419 (+ 301us) client_negotiation.cc:240] Negotiation successful
0810 19:35:05.006720 (+ 1301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35270,"thread_start_us":36831,"threads_started":1}
I20260810 19:35:05.009995 12069 heartbeater.cc:347] Connected to a master server at 127.29.233.190:36315
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
I20260810 19:35:05.010375 12069 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:05.011298 12069 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
I20260810 19:35:05.014379 11831 ts_manager.cc:194] Registered new tserver with Master: 9b694642140a43ef9221282873844de9 (127.29.233.129:38467)
I20260810 19:35:05.016449 11831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:54521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
W20260810 19:35:05.057714 30630 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.233.130: real 0.105s user 0.000s sys 0.025s
I20260810 19:35:05.058809 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:36315
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:35:05 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
Aug 10 19:35:05 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
W20260810 19:35:05.557225 12077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:05.557746 12077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:05.557974 12077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:05.568660 12077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:05.568885 12077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:05.569139 12077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:35:05.585088 12077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.233.148:36999/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:05.587036 12077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:05.588940 12077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:05.614775 12085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:05.618918 12086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:05.633236 12088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:05.633498 12077 server_base.cc:1034] running on GCE node
I20260810 19:35:05.634554 12077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:05.635954 12077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:05.637440 12077 hybrid_clock.cc:648] HybridClock initialized: now 1786390505637327 us; error 98 us; skew 500 ppm
I20260810 19:35:05.637916 12077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:05 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390505, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:05.642565 12077 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:35:05.658001 12077 webserver.cc:460] Webserver started at http://127.29.233.130:46075/ using document root <none> and password file <none>
I20260810 19:35:05.659291 12077 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:05.659540 12077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:05.660107 12077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:35:05 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390505, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:05.684749 12077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "a5261a972e1c4bf0bbbe47c26c34842f"
format_stamp: "Formatted at 2026-08-10 19:35:05 on dist-test-slave-tm1g"
server_key: "84c94c17dd0da9f34790d44881d46aa4"
server_key_iv: "2938ff8955322263a385c007f1c6a70a"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:05.686007 12077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "a5261a972e1c4bf0bbbe47c26c34842f"
format_stamp: "Formatted at 2026-08-10 19:35:05 on dist-test-slave-tm1g"
server_key: "84c94c17dd0da9f34790d44881d46aa4"
server_key_iv: "2938ff8955322263a385c007f1c6a70a"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:05.693367 12077 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260810 19:35:05.716375 12095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:05.719488 12077 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 19:35:05.719753 12077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "a5261a972e1c4bf0bbbe47c26c34842f"
format_stamp: "Formatted at 2026-08-10 19:35:05 on dist-test-slave-tm1g"
server_key: "84c94c17dd0da9f34790d44881d46aa4"
server_key_iv: "2938ff8955322263a385c007f1c6a70a"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:05.720098 12077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:05.763864 12077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:05.768227 12077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:05.768572 12077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:05.769615 12077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:05.771199 12077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:05.771306 12077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:05.771461 12077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:05.771533 12077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:05.812045 12077 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:33633
I20260810 19:35:05.812070 12227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:33633 every 8 connection(s)
I20260810 19:35:05.813611 12077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:35:05.818037 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 12077
I20260810 19:35:05.818222 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:35:05.818864 30630 external_mini_cluster.cc:1428] Setting key aee3663df72783d96dbafe62abfe408e
Aug 10 19:35:05 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390505, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:05.835103 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:05.816527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:54579 (local address 127.29.233.190:36315)
0810 19:35:05.817029 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:05.817036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:05.819569 (+ 2533us) server_negotiation.cc:407] Connection header received
0810 19:35:05.820135 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:05.820141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:05.820245 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:05.820378 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:05.822433 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:05.823196 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:05.824786 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:05.825245 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:05.825382 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:05.829060 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:05.829065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:05.829068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:05.829102 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:05.831962 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:05.832722 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:05.832729 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:05.832733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:05.832869 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:05.833288 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:05.833294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:05.833297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:05.833586 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:05.833774 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:05.834468 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 19:35:05.834707 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":237,"threads_started":1}
I20260810 19:35:05.836664 12233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:05.817118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:36315 (local address 127.29.233.130:54579)
0810 19:35:05.817925 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:05.817972 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:05.819850 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:05.820673 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:05.820685 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:05.821296 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:05.821894 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:05.821912 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:05.823352 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:05.823357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:05.824625 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:05.824637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:05.825529 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:05.825537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:05.825773 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:05.826673 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:05.826695 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:05.828835 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:05.832175 (+ 3340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:05.832183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:05.832197 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:05.832605 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:05.832987 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:05.832994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:05.832997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:05.833178 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:05.833727 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:05.833735 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:05.834077 (+ 342us) client_negotiation.cc:769] Sending connection context
0810 19:35:05.834395 (+ 318us) client_negotiation.cc:240] Negotiation successful
0810 19:35:05.834923 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":360,"threads_started":1}
I20260810 19:35:05.838436 12228 heartbeater.cc:347] Connected to a master server at 127.29.233.190:36315
I20260810 19:35:05.838770 12228 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:05.839600 12228 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
I20260810 19:35:05.841327 11831 ts_manager.cc:194] Registered new tserver with Master: a5261a972e1c4bf0bbbe47c26c34842f (127.29.233.130:33633)
I20260810 19:35:05.842649 11831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:54579
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:35:05.891692 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:36315
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:35:06.020448 12069 heartbeater.cc:503] Master 127.29.233.190:36315 was elected leader, sending a full tablet report...
W20260810 19:35:06.248023 12238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:06.248310 12238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:06.248409 12238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:06.255817 12238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:06.255928 12238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:06.256042 12238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:35:06.265933 12238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.233.148:36999/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:06.267355 12238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:06.268586 12238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:06.281749 12247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:06.281699 12244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:06.281847 12243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:06.282534 12238 server_base.cc:1034] running on GCE node
I20260810 19:35:06.283263 12238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:06.284219 12238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:06.285447 12238 hybrid_clock.cc:648] HybridClock initialized: now 1786390506285397 us; error 70 us; skew 500 ppm
I20260810 19:35:06.285789 12238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:06 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390506, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:06.289079 12238 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:35:06.290714 12238 webserver.cc:460] Webserver started at http://127.29.233.131:37843/ using document root <none> and password file <none>
I20260810 19:35:06.291419 12238 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:06.291567 12238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:06.291940 12238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 19:35:06 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390506, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:06.309126 12238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "d9ce9bbdc39f4bbda6197b13df1703ef"
format_stamp: "Formatted at 2026-08-10 19:35:06 on dist-test-slave-tm1g"
server_key: "6635234851ba42eb4ecf8bdcb3687f65"
server_key_iv: "4c3b3bb47bbd3b36f0721092431739c5"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:06.310235 12238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "d9ce9bbdc39f4bbda6197b13df1703ef"
format_stamp: "Formatted at 2026-08-10 19:35:06 on dist-test-slave-tm1g"
server_key: "6635234851ba42eb4ecf8bdcb3687f65"
server_key_iv: "4c3b3bb47bbd3b36f0721092431739c5"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:06.317569 12238 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260810 19:35:06.334697 12255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:06.336498 12238 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 19:35:06.336694 12238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "d9ce9bbdc39f4bbda6197b13df1703ef"
format_stamp: "Formatted at 2026-08-10 19:35:06 on dist-test-slave-tm1g"
server_key: "6635234851ba42eb4ecf8bdcb3687f65"
server_key_iv: "4c3b3bb47bbd3b36f0721092431739c5"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:06.336961 12238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:06.374558 12238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:06.378453 12238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:06.378772 12238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:06.379813 12238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:06.381222 12238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:06.381297 12238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:06.381480 12238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:06.381533 12238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:06.416566 12238 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:36241
I20260810 19:35:06.416599 12374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:36241 every 8 connection(s)
I20260810 19:35:06.417951 12238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:35:06.424675 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 12238
I20260810 19:35:06.424870 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:35:06.425339 30630 external_mini_cluster.cc:1428] Setting key 4c1f09627b9068c164e5a1f69942554f
Aug 10 19:35:06 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390506, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:06.444262 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:06.420707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:54867 (local address 127.29.233.190:36315)
0810 19:35:06.421218 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:06.421223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:06.422187 (+ 964us) server_negotiation.cc:407] Connection header received
0810 19:35:06.424023 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:06.424028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:06.424146 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:06.424272 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:06.426013 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.427064 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.428977 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.429480 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.429614 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:06.434625 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:06.434631 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:06.434634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:06.434679 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:06.438949 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.440096 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.440100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.440103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.440415 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.441214 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.441220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.441222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.441938 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:06.442160 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:06.443518 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0810 19:35:06.443853 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":241,"threads_started":1}
I20260810 19:35:06.446038 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:06.421328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:36315 (local address 127.29.233.131:54867)
0810 19:35:06.422026 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:06.422062 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:06.423756 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:06.424532 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:06.424540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:06.425061 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:06.425840 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:06.425859 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.427213 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.427218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:06.428819 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:06.428831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.429711 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.429720 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:06.429957 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:06.430939 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:06.430965 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:06.434215 (+ 3250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:06.439212 (+ 4997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:06.439222 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:06.439265 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:06.439878 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:06.440597 (+ 719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:06.440615 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:06.440620 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:06.441042 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:06.442122 (+ 1080us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:06.442129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:06.442858 (+ 729us) client_negotiation.cc:769] Sending connection context
0810 19:35:06.443315 (+ 457us) client_negotiation.cc:240] Negotiation successful
0810 19:35:06.443981 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":287,"threads_started":1}
I20260810 19:35:06.448186 12375 heartbeater.cc:347] Connected to a master server at 127.29.233.190:36315
I20260810 19:35:06.448799 12375 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:06.450013 12375 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
I20260810 19:35:06.452253 11831 ts_manager.cc:194] Registered new tserver with Master: d9ce9bbdc39f4bbda6197b13df1703ef (127.29.233.131:36241)
I20260810 19:35:06.454008 11831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:54867
I20260810 19:35:06.455960 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:35:06.458818 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 11921
Aug 10 19:35:06 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:35:06.488104 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:06.460510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.29.233.129:38467)
0810 19:35:06.461601 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:06.461610 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:06.461652 (+ 42us) server_negotiation.cc:407] Connection header received
0810 19:35:06.461737 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:06.461743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:06.462067 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:06.462226 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:06.464680 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.466460 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.469723 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.470833 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.471019 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:06.478652 (+ 7633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:06.478674 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:06.478699 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:06.478762 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:06.483655 (+ 4893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.484780 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.484788 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.484791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.485039 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.485657 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.485660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.485662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.486287 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:06.486745 (+ 458us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:06.487155 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 19:35:06.487486 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":594,"threads_started":1}
I20260810 19:35:06.490196 12064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:06.617270 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 11921
I20260810 19:35:06.652391 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 12077
Aug 10 19:35:06 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:35:06.680739 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:06.653956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.29.233.130:33633)
0810 19:35:06.654952 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:06.654972 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:06.655018 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:35:06.655128 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:06.655144 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:06.655545 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:06.655758 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:06.657982 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.659938 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.662223 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.663256 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.663442 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:06.671451 (+ 8009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:06.671473 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:06.671499 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:06.671563 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:06.675938 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.677045 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.677054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.677057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.677270 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.677889 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.677892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.677894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.678584 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:06.678952 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:06.679649 (+ 697us) server_negotiation.cc:299] Negotiation successful
0810 19:35:06.680001 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":572,"threads_started":1}
I20260810 19:35:06.683090 12223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:06.794692 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 12077
I20260810 19:35:06.815364 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 12238
Aug 10 19:35:06 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:35:06.832767 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:06.816469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.29.233.131:36241)
0810 19:35:06.817125 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:06.817132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:06.817161 (+ 29us) server_negotiation.cc:407] Connection header received
0810 19:35:06.817219 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:06.817225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:06.817446 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:06.817571 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:06.819061 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.820116 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.821412 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:06.821971 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:06.822070 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:06.826810 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:06.826822 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:06.826838 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:06.826873 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:06.829742 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.830574 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.830581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.830583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.830712 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:06.831074 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:06.831077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:06.831079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:06.831424 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:06.831776 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:06.832113 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 19:35:06.832383 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":320,"threads_started":1}
I20260810 19:35:06.834167 12368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:06.921900 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 12238
I20260810 19:35:06.942751 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 11796
I20260810 19:35:06.944144 11864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:07.077204 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 11796
I20260810 19:35:07.103291 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:36315
--webserver_interface=127.29.233.190
--webserver_port=42313
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:36315
--ranger_config_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:07.442744 12388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:07.443049 12388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:07.443226 12388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:07.450996 12388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:07.451120 12388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:07.451169 12388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:07.451234 12388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:07.451283 12388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:07.462338 12388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:36315
--ranger_config_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:36315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:07.463670 12388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:07.464969 12388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:07.477468 12397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:07.477573 12395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:07.477808 12393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:07.478600 12388 server_base.cc:1034] running on GCE node
I20260810 19:35:07.479387 12388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:07.480844 12388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:07.482088 12388 hybrid_clock.cc:648] HybridClock initialized: now 1786390507482027 us; error 53 us; skew 500 ppm
I20260810 19:35:07.482530 12388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:07 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390507, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:07.485713 12388 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:07.487319 12388 webserver.cc:460] Webserver started at http://127.29.233.190:42313/ using document root <none> and password file <none>
I20260810 19:35:07.488018 12388 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:07.488174 12388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 19:35:07 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390507, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:07.574879 12388 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 19:35:07.578003 12405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:07.579437 12388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:07.579670 12388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "c6dab78496e24ad08195e57d347c166f"
format_stamp: "Formatted at 2026-08-10 19:35:03 on dist-test-slave-tm1g"
server_key: "1285c45908d097356bbe61d783b40713"
server_key_iv: "345fff8d8479b8591cac0d470b8e6986"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:07.580662 12388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:07.596729 12388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:07.600381 12388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:07.600710 12388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:07.619894 12388 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:36315
I20260810 19:35:07.619920 12461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:36315 every 8 connection(s)
I20260810 19:35:07.621440 12388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:07.623370 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 12388
I20260810 19:35:07.624445 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:38467
--local_ip_for_outbound_sockets=127.29.233.129
--tserver_master_addrs=127.29.233.190:36315
--webserver_port=35269
--webserver_interface=127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:35:07.631641 12462 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:35:07.636852 12462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Bootstrap starting.
I20260810 19:35:07.656342 12462 log.cc:824] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:07.665426 12462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:35:07.665978 12462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Bootstrap complete.
I20260810 19:35:07.675618 12462 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:07.676430 12462 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6dab78496e24ad08195e57d347c166f, State: Initialized, Role: FOLLOWER
I20260810 19:35:07.677099 12462 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:07.677326 12462 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:35:07.677472 12462 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:35:07.677664 12462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 1 FOLLOWER]: Advancing to term 2
I20260810 19:35:07.681504 12462 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:07.681993 12462 leader_election.cc:302] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6dab78496e24ad08195e57d347c166f; no voters:
I20260810 19:35:07.682662 12462 leader_election.cc:288] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 19:35:07.682811 12466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 2 FOLLOWER]: Leader election won for term 2
I20260810 19:35:07.684087 12466 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [term 2 LEADER]: Becoming Leader. State: Replica: c6dab78496e24ad08195e57d347c166f, State: Running, Role: LEADER
I20260810 19:35:07.684758 12466 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } }
I20260810 19:35:07.685108 12462 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:07.688033 12468 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6dab78496e24ad08195e57d347c166f. Latest consensus state: current_term: 2 leader_uuid: "c6dab78496e24ad08195e57d347c166f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } } }
I20260810 19:35:07.690238 12468 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:07.689790 12467 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c6dab78496e24ad08195e57d347c166f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dab78496e24ad08195e57d347c166f" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 36315 } } }
I20260810 19:35:07.690945 12467 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:07.701612 12473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:07.707474 12473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:07.708362 12462 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 19:35:07.708966 12473 catalog_manager.cc:1259] Loaded cluster ID: 0936e7f38be2477aa040e3109669c771
I20260810 19:35:07.709049 12473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:35:07.720587 12473 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:35:07.723397 12473 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Loaded TSK: 0
I20260810 19:35:07.725525 12473 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 19:35:08.102782 12464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:08.103222 12464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:08.103463 12464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:08.115499 12464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:08.115711 12464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:08.115935 12464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:35:08.132230 12464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:38467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=35269
--tserver_master_addrs=127.29.233.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:08.134356 12464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:08.136314 12464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:08.182077 12507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:08.182717 12504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:08.183651 12503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:08.187444 12464 server_base.cc:1034] running on GCE node
I20260810 19:35:08.188953 12464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:08.190591 12464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:08.192148 12464 hybrid_clock.cc:648] HybridClock initialized: now 1786390508191847 us; error 322 us; skew 500 ppm
I20260810 19:35:08.192615 12464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:08 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:08.201923 12464 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:35:08.225656 12464 webserver.cc:460] Webserver started at http://127.29.233.129:35269/ using document root <none> and password file <none>
I20260810 19:35:08.226675 12464 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:08.226868 12464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 19:35:08 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:08.260453 12464 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.007s
I20260810 19:35:08.276935 12515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:08.279199 12464 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.001s
I20260810 19:35:08.279484 12464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "9b694642140a43ef9221282873844de9"
format_stamp: "Formatted at 2026-08-10 19:35:04 on dist-test-slave-tm1g"
server_key: "a07da0f4ce8469f8bb47ca81a8f325ed"
server_key_iv: "17e031e09c3ccef9dc98a7f3ba637e23"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:08.281113 12464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:08.320987 12464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:08.326259 12464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:08.326664 12464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:08.328058 12464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:08.329962 12464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:08.330101 12464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:08.330216 12464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:08.330333 12464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:08.391358 12464 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:38467
I20260810 19:35:08.391417 12639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:38467 every 8 connection(s)
I20260810 19:35:08.393154 12464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:35:08.401508 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 12464
I20260810 19:35:08.402565 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:33633
--local_ip_for_outbound_sockets=127.29.233.130
--tserver_master_addrs=127.29.233.190:36315
--webserver_port=46075
--webserver_interface=127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:35:08 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:08.446599 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:08.397823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:36315 (local address 127.29.233.129:35189)
0810 19:35:08.408707 (+ 10884us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:08.408768 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:08.411475 (+ 2707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:08.415062 (+ 3587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:08.415074 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:08.415811 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:08.416748 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:08.416777 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:08.419431 (+ 2654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:08.419436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:08.422053 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:08.422069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:08.424012 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:08.424021 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:08.425439 (+ 1418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:08.426290 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:08.426360 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:08.432559 (+ 6199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:08.440835 (+ 8276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:08.440847 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:08.440865 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:08.441567 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:08.442220 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:08.442225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:08.442229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:08.442542 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:08.443746 (+ 1204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:08.443756 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:08.444186 (+ 430us) client_negotiation.cc:769] Sending connection context
0810 19:35:08.444490 (+ 304us) client_negotiation.cc:240] Negotiation successful
0810 19:35:08.445099 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10502,"thread_start_us":10186,"threads_started":1}
I20260810 19:35:08.448984 12643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:08.399066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:35189 (local address 127.29.233.190:36315)
0810 19:35:08.407881 (+ 8815us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:08.407907 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:08.409185 (+ 1278us) server_negotiation.cc:407] Connection header received
0810 19:35:08.412455 (+ 3270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:08.412478 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:08.413762 (+ 1284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:08.414230 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:08.417110 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:08.419224 (+ 2114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:08.422253 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:08.423381 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:08.423583 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:08.432957 (+ 9374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:08.432971 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:08.433007 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:08.433073 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:08.438350 (+ 5277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:08.441797 (+ 3447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:08.441809 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:08.441817 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:08.442048 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:08.442714 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:08.442720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:08.442724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:08.443444 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:08.443653 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:08.447336 (+ 3683us) server_negotiation.cc:299] Negotiation successful
0810 19:35:08.447785 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8463,"thread_start_us":8208,"threads_started":1}
I20260810 19:35:08.453394 12640 heartbeater.cc:347] Connected to a master server at 127.29.233.190:36315
I20260810 19:35:08.453755 12640 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:08.454814 12640 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
W20260810 19:35:08.458184 12640 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:36315 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 19:35:08.458436 12640 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:08.458722 12640 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
I20260810 19:35:08.460052 12640 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:08.460389 12640 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
W20260810 19:35:08.461480 12640 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 19:35:08.823612 12644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:08.824100 12644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:08.824290 12644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:08.833684 12644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:08.833904 12644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:08.834141 12644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:35:08.850142 12644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:33633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=46075
--tserver_master_addrs=127.29.233.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:08.852083 12644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:08.853889 12644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:08.868783 12659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:08.872462 12660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:08.879230 12644 server_base.cc:1034] running on GCE node
W20260810 19:35:08.879824 12665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:08.880764 12644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:08.886662 12644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:08.887979 12644 hybrid_clock.cc:648] HybridClock initialized: now 1786390508887867 us; error 107 us; skew 500 ppm
I20260810 19:35:08.888492 12644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:08 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:08.893181 12644 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:35:08.907898 12644 webserver.cc:460] Webserver started at http://127.29.233.130:46075/ using document root <none> and password file <none>
I20260810 19:35:08.908747 12644 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:08.908941 12644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 19:35:08 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:08.951478 12644 fs_manager.cc:709] Time spent opening directory manager: real 0.024s user 0.009s sys 0.011s
Aug 10 19:35:08 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:08.961840 12676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:08.963941 12644 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.006s
I20260810 19:35:08.964185 12644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "a5261a972e1c4bf0bbbe47c26c34842f"
format_stamp: "Formatted at 2026-08-10 19:35:05 on dist-test-slave-tm1g"
server_key: "84c94c17dd0da9f34790d44881d46aa4"
server_key_iv: "2938ff8955322263a385c007f1c6a70a"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:08.965842 12644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:09.010761 12644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:09.016434 12644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:09.016847 12644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:09.024174 12644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:09.026249 12644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:09.026412 12644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 19:35:09.026556 12644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:09.026634 12644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:09.079137 12644 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:33633
I20260810 19:35:09.079151 12801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:33633 every 8 connection(s)
I20260810 19:35:09.081081 12644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:35:09.084442 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 12644
I20260810 19:35:09.085282 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.233.148:36999/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:36241
--local_ip_for_outbound_sockets=127.29.233.131
--tserver_master_addrs=127.29.233.190:36315
--webserver_port=37843
--webserver_interface=127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:35:09 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:09.133328 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:09.084005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:46145 (local address 127.29.233.190:36315)
0810 19:35:09.106967 (+ 22962us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:09.106975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:09.108228 (+ 1253us) server_negotiation.cc:407] Connection header received
0810 19:35:09.110839 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:09.110846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:09.110955 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:09.111072 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:09.113392 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.114635 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.117123 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.117773 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.117881 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:09.125572 (+ 7691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:09.125580 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:09.125585 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:09.125631 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:09.129411 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:09.130406 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:09.130413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:09.130417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:09.130583 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:09.131170 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:09.131174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:09.131178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:09.131578 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:09.131751 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:09.132653 (+ 902us) server_negotiation.cc:299] Negotiation successful
0810 19:35:09.132899 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22791,"thread_start_us":22637,"threads_started":1}
I20260810 19:35:09.134534 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:09.084616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:36315 (local address 127.29.233.130:46145)
0810 19:35:09.107676 (+ 23060us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:09.107757 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:09.110346 (+ 2589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:09.111579 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:09.111593 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:09.112251 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:09.113145 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:09.113172 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.114848 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.114853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:09.116942 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:09.116958 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.118287 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.118296 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:09.118652 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:09.121535 (+ 2883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:09.121566 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:09.125223 (+ 3657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:09.129566 (+ 4343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:09.129576 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:09.129595 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:09.130223 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:09.130710 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:09.130715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:09.130719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:09.131023 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:09.131735 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:09.131744 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:09.132207 (+ 463us) client_negotiation.cc:769] Sending connection context
0810 19:35:09.132517 (+ 310us) client_negotiation.cc:240] Negotiation successful
0810 19:35:09.133092 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22676,"thread_start_us":22407,"threads_started":1}
I20260810 19:35:09.136323 12802 heartbeater.cc:347] Connected to a master server at 127.29.233.190:36315
I20260810 19:35:09.136641 12802 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:09.137326 12802 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
I20260810 19:35:09.139325 12423 ts_manager.cc:194] Registered new tserver with Master: a5261a972e1c4bf0bbbe47c26c34842f (127.29.233.130:33633)
I20260810 19:35:09.141414 12423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:46145
I20260810 19:35:09.461669 12640 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:09.462153 12640 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
I20260810 19:35:09.463755 12423 ts_manager.cc:194] Registered new tserver with Master: 9b694642140a43ef9221282873844de9 (127.29.233.129:38467)
I20260810 19:35:09.464991 12423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:35189
Aug 10 19:35:09 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
W20260810 19:35:09.576704 12807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:09.577171 12807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:09.577329 12807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:09.588451 12807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:09.588653 12807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:09.588835 12807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:35:09.604717 12807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.233.148:36999/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:36241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=37843
--tserver_master_addrs=127.29.233.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:09.606177 12807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:09.607734 12807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:09.623131 12812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:09.632527 12813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:09.648259 12815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:09.648948 12807 server_base.cc:1034] running on GCE node
I20260810 19:35:09.649856 12807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:09.651209 12807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:09.652487 12807 hybrid_clock.cc:648] HybridClock initialized: now 1786390509652431 us; error 44 us; skew 500 ppm
I20260810 19:35:09.652952 12807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:09 dist-test-slave-tm1g krb5kdc[11299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:09.656483 12807 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:35:09.660583 12807 webserver.cc:460] Webserver started at http://127.29.233.131:37843/ using document root <none> and password file <none>
I20260810 19:35:09.661366 12807 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:09.661531 12807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 19:35:09 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:09.706537 12807 fs_manager.cc:709] Time spent opening directory manager: real 0.027s user 0.009s sys 0.018s
I20260810 19:35:09.711634 12824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:09.713526 12807 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:35:09.713713 12807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "d9ce9bbdc39f4bbda6197b13df1703ef"
format_stamp: "Formatted at 2026-08-10 19:35:06 on dist-test-slave-tm1g"
server_key: "6635234851ba42eb4ecf8bdcb3687f65"
server_key_iv: "4c3b3bb47bbd3b36f0721092431739c5"
server_key_version: "kuduclusterkey@0"
I20260810 19:35:09.715204 12807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:09.738759 12807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:09.755431 12807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:09.755836 12807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:09.757113 12807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:09.758971 12807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:09.759189 12807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:09.759344 12807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:09.759478 12807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 19:35:09 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390508, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for HTTP/127.29.233.148@KRBTEST.COM
I20260810 19:35:09.946548 12807 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:36241
I20260810 19:35:09.946604 12956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:36241 every 8 connection(s)
I20260810 19:35:09.948676 12807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:35:09.955084 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 12807
Aug 10 19:35:09 dist-test-slave-tm1g krb5kdc[11299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:10.000731 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:09.956872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38602 (local address 127.29.233.190:36315)
0810 19:35:09.975721 (+ 18849us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:09.975728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:09.975750 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:35:09.975828 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:09.975834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:09.975984 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:09.976158 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:09.978585 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.979791 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.984298 (+ 4507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.985040 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.988755 (+ 3715us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:09.988987 (+ 232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:09.988993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:09.988997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:09.989041 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:09.995368 (+ 6327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:09.997379 (+ 2011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:09.997385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:09.997389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:09.997561 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:09.998078 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:09.998083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:09.998087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:09.998496 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:09.999936 (+ 1440us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.000086 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.000330 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18725,"thread_start_us":18620,"threads_started":1}
I20260810 19:35:10.012218 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:09.960966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:36315 (local address 127.29.233.131:46465)
0810 19:35:09.973755 (+ 12789us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:09.973827 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:09.981161 (+ 7334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:09.988111 (+ 6950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:09.988127 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:09.988924 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:09.989803 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:09.989828 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.993201 (+ 3373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.993208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:09.995142 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:09.995157 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.996471 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.996478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:09.996808 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:09.998525 (+ 1717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:09.998556 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:10.003017 (+ 4461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:10.007026 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:10.007035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:10.007054 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:10.007735 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:10.008218 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:10.008223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:10.008236 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:10.008530 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:10.009296 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:10.009305 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:10.009784 (+ 479us) client_negotiation.cc:769] Sending connection context
0810 19:35:10.010126 (+ 342us) client_negotiation.cc:240] Negotiation successful
0810 19:35:10.010733 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12333,"thread_start_us":13209,"threads_started":1}
I20260810 19:35:10.013640 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:09.975698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:46465 (local address 127.29.233.190:36315)
0810 19:35:09.987067 (+ 11369us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:09.987073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:09.987092 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:35:09.987146 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:09.987152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:09.987488 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:09.987605 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:09.990915 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.992187 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.995347 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:09.996012 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:09.996142 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.003310 (+ 7168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:10.003317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:10.003321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:10.003363 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:10.006849 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.007912 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.007917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.007921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.008077 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.008687 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.008692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.008695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.009063 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:10.009203 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.013017 (+ 3814us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.013276 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11236,"thread_start_us":11566,"threads_started":1}
I20260810 19:35:10.015596 12957 heartbeater.cc:347] Connected to a master server at 127.29.233.190:36315
I20260810 19:35:10.015918 12957 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:10.016840 12957 heartbeater.cc:511] Master 127.29.233.190:36315 requested a full tablet report, sending...
I20260810 19:35:10.018887 12423 ts_manager.cc:194] Registered new tserver with Master: d9ce9bbdc39f4bbda6197b13df1703ef (127.29.233.131:36241)
I20260810 19:35:10.019757 12423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:46465
I20260810 19:35:10.031716 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:35:10.059619 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.045078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.29.233.190:36315)
0810 19:35:10.045327 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.045336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.045915 (+ 579us) server_negotiation.cc:407] Connection header received
0810 19:35:10.046150 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.046156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.046242 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.046382 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:10.047973 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.048716 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.050094 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.050645 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.050788 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.053739 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:10.053744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:10.053748 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:10.053783 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:10.056540 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.057254 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.057260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.057264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.057429 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.057916 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.057922 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.057926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.058329 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:10.058563 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.059030 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.059274 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:35:10.065542 12423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:35:10.077700 12423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:35:10.112430 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.100623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34888)
0810 19:35:10.101717 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.101781 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:10.101993 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:10.103898 (+ 1905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:10.103910 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:10.103950 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:10.104504 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.104521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.106652 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.106658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.109081 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.109097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.111342 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.111348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.111571 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.111625 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 19:35:10.111757 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 19:35:10.111944 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":9993,"threads_started":1}
I20260810 19:35:10.113759 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.101296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.29.233.129:38467)
0810 19:35:10.103254 (+ 1958us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.103262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.103289 (+ 27us) server_negotiation.cc:407] Connection header received
0810 19:35:10.103400 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.103408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.103627 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.103828 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:10.104740 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.106483 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.109295 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.110826 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.111017 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.111184 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.112908 (+ 1724us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.113085 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1816,"thread_start_us":1649,"threads_started":1}
I20260810 19:35:10.117980 12568 tablet_service.cc:1467] Processing CreateTablet for tablet 678836230fe6473c930e71390257514f (DEFAULT_TABLE table=test-table [id=3c071ce46a0b4e73ae4acd376fb88d6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:10.119601 12568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678836230fe6473c930e71390257514f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:10.121227 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.111323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.29.233.131:36241)
0810 19:35:10.112499 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.112509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.113012 (+ 503us) server_negotiation.cc:407] Connection header received
0810 19:35:10.113112 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.113118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.113328 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.113535 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:10.114254 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.115735 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.118394 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.119735 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.119918 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.120121 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.120672 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.120834 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1000,"thread_start_us":853,"threads_started":1}
I20260810 19:35:10.121258 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.112505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:36241 (local address 127.0.0.1:49876)
0810 19:35:10.112768 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.112784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:10.112896 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:10.113829 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:10.113835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:10.113861 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:10.114057 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.114069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.115879 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.115885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.118137 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.118148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.120230 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.120236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.120396 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.120505 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 19:35:10.120619 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 19:35:10.120852 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":201,"thread_start_us":1878,"threads_started":1}
I20260810 19:35:10.121819 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.101639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.29.233.130:33633)
0810 19:35:10.102779 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.102791 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.111023 (+ 8232us) server_negotiation.cc:407] Connection header received
0810 19:35:10.111349 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.111357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.111669 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.111871 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:10.113036 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.115547 (+ 2511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.118350 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.119827 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.120043 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.120199 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.121268 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.121444 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":987,"threads_started":1}
I20260810 19:35:10.122155 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.100861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33633 (local address 127.0.0.1:36634)
0810 19:35:10.110899 (+ 10038us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.110920 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:10.111142 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:10.112225 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:10.112230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:10.112254 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:10.112745 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.112764 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.115728 (+ 2964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.115732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.118133 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.118148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.120386 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.120392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.120584 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.120619 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 19:35:10.120714 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 19:35:10.120851 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9897,"mutex_wait_us":122,"thread_start_us":1055,"threads_started":1}
I20260810 19:35:10.125203 12881 tablet_service.cc:1467] Processing CreateTablet for tablet 678836230fe6473c930e71390257514f (DEFAULT_TABLE table=test-table [id=3c071ce46a0b4e73ae4acd376fb88d6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:10.125674 12729 tablet_service.cc:1467] Processing CreateTablet for tablet 678836230fe6473c930e71390257514f (DEFAULT_TABLE table=test-table [id=3c071ce46a0b4e73ae4acd376fb88d6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:10.127133 12881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678836230fe6473c930e71390257514f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:10.127148 12729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678836230fe6473c930e71390257514f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:10.132525 12986 tablet_bootstrap.cc:492] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: Bootstrap starting.
I20260810 19:35:10.136353 12986 tablet_bootstrap.cc:654] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:10.137441 12986 log.cc:824] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:10.140139 12986 tablet_bootstrap.cc:492] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: No bootstrap required, opened a new log
I20260810 19:35:10.140415 12986 ts_tablet_manager.cc:1397] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260810 19:35:10.143603 12989 tablet_bootstrap.cc:492] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Bootstrap starting.
I20260810 19:35:10.144405 12988 tablet_bootstrap.cc:492] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Bootstrap starting.
I20260810 19:35:10.145642 12802 heartbeater.cc:503] Master 127.29.233.190:36315 was elected leader, sending a full tablet report...
I20260810 19:35:10.148104 12989 tablet_bootstrap.cc:654] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:10.149389 12989 log.cc:824] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:10.149995 12988 tablet_bootstrap.cc:654] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:10.150668 12986 raft_consensus.cc:348] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.151075 12988 log.cc:824] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:10.151237 12986 raft_consensus.cc:374] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:10.151365 12986 raft_consensus.cc:729] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b694642140a43ef9221282873844de9, State: Initialized, Role: FOLLOWER
I20260810 19:35:10.152245 12986 consensus_queue.cc:260] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.154582 12988 tablet_bootstrap.cc:492] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: No bootstrap required, opened a new log
I20260810 19:35:10.154870 12988 ts_tablet_manager.cc:1397] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260810 19:35:10.157080 12640 heartbeater.cc:503] Master 127.29.233.190:36315 was elected leader, sending a full tablet report...
I20260810 19:35:10.157802 12986 ts_tablet_manager.cc:1428] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: Time spent starting tablet: real 0.017s user 0.009s sys 0.008s
I20260810 19:35:10.160224 12989 tablet_bootstrap.cc:492] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: No bootstrap required, opened a new log
I20260810 19:35:10.160537 12989 ts_tablet_manager.cc:1397] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260810 19:35:10.163493 12988 raft_consensus.cc:348] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.163854 12988 raft_consensus.cc:374] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:10.163986 12988 raft_consensus.cc:729] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ce9bbdc39f4bbda6197b13df1703ef, State: Initialized, Role: FOLLOWER
I20260810 19:35:10.164539 12988 consensus_queue.cc:260] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.165601 12957 heartbeater.cc:503] Master 127.29.233.190:36315 was elected leader, sending a full tablet report...
I20260810 19:35:10.167346 12988 ts_tablet_manager.cc:1428] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260810 19:35:10.169916 12989 raft_consensus.cc:348] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.170286 12989 raft_consensus.cc:374] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:10.170444 12989 raft_consensus.cc:729] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5261a972e1c4bf0bbbe47c26c34842f, State: Initialized, Role: FOLLOWER
I20260810 19:35:10.170957 12989 consensus_queue.cc:260] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.172644 12989 ts_tablet_manager.cc:1428] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260810 19:35:10.200610 12992 raft_consensus.cc:482] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:35:10.201200 12992 raft_consensus.cc:504] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.203188 12992 leader_election.cc:288] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5261a972e1c4bf0bbbe47c26c34842f (127.29.233.130:33633), d9ce9bbdc39f4bbda6197b13df1703ef (127.29.233.131:36241)
I20260810 19:35:10.210358 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.203729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:59419 (local address 127.29.233.130:33633)
0810 19:35:10.203967 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.203986 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.204909 (+ 923us) server_negotiation.cc:407] Connection header received
0810 19:35:10.205108 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.205113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.205180 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.205283 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:10.205982 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.206772 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.208612 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.209390 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.209574 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.209651 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.209947 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.210073 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:35:10.210522 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.203964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:52797 (local address 127.29.233.131:36241)
0810 19:35:10.204211 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.204219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.204689 (+ 470us) server_negotiation.cc:407] Connection header received
0810 19:35:10.204922 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.204926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.204987 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.205085 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:10.205777 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.206587 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.208547 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.209322 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.209513 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.209594 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.210130 (+ 536us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.210256 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:35:10.210551 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.203723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33633 (local address 127.29.233.129:59419)
0810 19:35:10.204787 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.204804 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:10.204957 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:10.205331 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:10.205336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:10.205400 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:10.205805 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.205813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.206915 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.206926 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.208453 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.208466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.209614 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.209621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.209779 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.209803 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:35:10.209921 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 19:35:10.210050 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":44,"spinlock_wait_cycles":7296,"thread_start_us":454,"threads_started":1}
I20260810 19:35:10.210719 12996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.203438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:36241 (local address 127.29.233.129:52797)
0810 19:35:10.204564 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.204583 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:10.204759 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:10.205108 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:10.205113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:10.205140 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:10.205614 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.205624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.206716 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.206724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.208383 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:10.208396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.209563 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.209571 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:10.209958 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.209983 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 19:35:10.210103 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 19:35:10.210236 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"mutex_wait_us":44,"spinlock_wait_cycles":4480,"thread_start_us":1024,"threads_started":1}
I20260810 19:35:10.211741 12753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678836230fe6473c930e71390257514f" candidate_uuid: "9b694642140a43ef9221282873844de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" is_pre_election: true
I20260810 19:35:10.212087 12903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678836230fe6473c930e71390257514f" candidate_uuid: "9b694642140a43ef9221282873844de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" is_pre_election: true
I20260810 19:35:10.212255 12753 raft_consensus.cc:2454] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b694642140a43ef9221282873844de9 in term 0.
I20260810 19:35:10.212560 12903 raft_consensus.cc:2454] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b694642140a43ef9221282873844de9 in term 0.
I20260810 19:35:10.212965 12518 leader_election.cc:302] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b694642140a43ef9221282873844de9, a5261a972e1c4bf0bbbe47c26c34842f; no voters:
I20260810 19:35:10.213446 12992 raft_consensus.cc:2781] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:35:10.213594 12992 raft_consensus.cc:482] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:35:10.213694 12992 raft_consensus.cc:3037] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:10.216055 12992 raft_consensus.cc:504] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
I20260810 19:35:10.216646 12992 leader_election.cc:288] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [CANDIDATE]: Term 1 election: Requested vote from peers a5261a972e1c4bf0bbbe47c26c34842f (127.29.233.130:33633), d9ce9bbdc39f4bbda6197b13df1703ef (127.29.233.131:36241)
I20260810 19:35:10.217104 12903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678836230fe6473c930e71390257514f" candidate_uuid: "9b694642140a43ef9221282873844de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef"
I20260810 19:35:10.217121 12753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678836230fe6473c930e71390257514f" candidate_uuid: "9b694642140a43ef9221282873844de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5261a972e1c4bf0bbbe47c26c34842f"
I20260810 19:35:10.217320 12753 raft_consensus.cc:3037] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:10.217307 12903 raft_consensus.cc:3037] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:10.219662 12753 raft_consensus.cc:2454] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b694642140a43ef9221282873844de9 in term 1.
I20260810 19:35:10.219669 12903 raft_consensus.cc:2454] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b694642140a43ef9221282873844de9 in term 1.
I20260810 19:35:10.220134 12520 leader_election.cc:302] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b694642140a43ef9221282873844de9, d9ce9bbdc39f4bbda6197b13df1703ef; no voters:
I20260810 19:35:10.220558 12992 raft_consensus.cc:2781] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:10.221007 12992 raft_consensus.cc:686] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 1 LEADER]: Becoming Leader. State: Replica: 9b694642140a43ef9221282873844de9, State: Running, Role: LEADER
I20260810 19:35:10.221571 12992 consensus_queue.cc:237] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } }
W20260810 19:35:10.225585 12961 tablet.cc:2369] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:35:10.226464 12423 catalog_manager.cc:5495] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b694642140a43ef9221282873844de9 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "9b694642140a43ef9221282873844de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b694642140a43ef9221282873844de9" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 38467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 } health_report { overall_health: UNKNOWN } } }
I20260810 19:35:10.268826 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.263315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.29.233.129:38467)
0810 19:35:10.263549 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.263555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.263573 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:35:10.263674 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.263678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.263763 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.263925 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:35:10.264319 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.265099 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.266577 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.266987 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.267125 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.267631 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:35:10.268197 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:35:10.268367 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.268476 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.268559 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:35:10.277086 12753 raft_consensus.cc:1261] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 1 FOLLOWER]: Refusing update from remote peer 9b694642140a43ef9221282873844de9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:10.277119 12903 raft_consensus.cc:1261] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 1 FOLLOWER]: Refusing update from remote peer 9b694642140a43ef9221282873844de9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:10.278100 12997 consensus_queue.cc:1035] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5261a972e1c4bf0bbbe47c26c34842f" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:10.278515 12992 consensus_queue.cc:1035] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9ce9bbdc39f4bbda6197b13df1703ef" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:10.290180 13000 mvcc.cc:205] Tried to move back new op lower bound from 7317055530085236736 to 7317055529871527936. Current Snapshot: MvccSnapshot[applied={T|T < 7317055530085236736}]
I20260810 19:35:10.303146 12423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:10.303459 12423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:10.307296 12423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6dab78496e24ad08195e57d347c166f: Sending DeleteTablet for 3 replicas of tablet 678836230fe6473c930e71390257514f
I20260810 19:35:10.308557 12729 tablet_service.cc:1514] Processing DeleteTablet for tablet 678836230fe6473c930e71390257514f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:10 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:36634
I20260810 19:35:10.308801 12881 tablet_service.cc:1514] Processing DeleteTablet for tablet 678836230fe6473c930e71390257514f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:10 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:49876
I20260810 19:35:10.309293 12568 tablet_service.cc:1514] Processing DeleteTablet for tablet 678836230fe6473c930e71390257514f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:10 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:34888
I20260810 19:35:10.317983 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 12464
I20260810 19:35:10.321740 13012 tablet_replica.cc:330] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: stopping tablet replica
I20260810 19:35:10.322358 13012 raft_consensus.cc:2229] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:10.323068 13012 raft_consensus.cc:2258] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:10.323724 13013 tablet_replica.cc:330] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: stopping tablet replica
I20260810 19:35:10.324357 13011 tablet_replica.cc:330] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: stopping tablet replica
I20260810 19:35:10.324751 13011 raft_consensus.cc:2229] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:10.325176 13011 raft_consensus.cc:2258] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:10.326148 13013 raft_consensus.cc:2229] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:35:10.327033 13013 raft_consensus.cc:2258] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:10.328819 13011 ts_tablet_manager.cc:1905] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:10.328819 13012 ts_tablet_manager.cc:1905] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:10.333328 13013 ts_tablet_manager.cc:1905] T 678836230fe6473c930e71390257514f P 9b694642140a43ef9221282873844de9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:10.336625 13012 ts_tablet_manager.cc:1918] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:10.336874 13012 log.cc:1196] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/678836230fe6473c930e71390257514f
I20260810 19:35:10.337550 13012 ts_tablet_manager.cc:1939] T 678836230fe6473c930e71390257514f P d9ce9bbdc39f4bbda6197b13df1703ef: Deleting consensus metadata
I20260810 19:35:10.339195 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.319245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.29.233.129:38467)
0810 19:35:10.320240 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.320246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.320265 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:35:10.320326 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.320331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.320439 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.320559 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:10.322840 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.324187 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.326389 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.327132 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.327267 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.331314 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:10.331329 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:10.331346 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:10.331413 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:10.335709 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.336660 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.336669 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.336672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.336897 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.337349 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.337355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.337358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.337890 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:10.338224 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.338576 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.338827 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842}
I20260810 19:35:10.339627 12409 catalog_manager.cc:4841] TS d9ce9bbdc39f4bbda6197b13df1703ef (127.29.233.131:36241): tablet 678836230fe6473c930e71390257514f (table test-table [id=3c071ce46a0b4e73ae4acd376fb88d6c]) successfully deleted
I20260810 19:35:10.340135 12635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:10.344637 13011 ts_tablet_manager.cc:1918] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:10.344831 13011 log.cc:1196] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/678836230fe6473c930e71390257514f
I20260810 19:35:10.345359 13011 ts_tablet_manager.cc:1939] T 678836230fe6473c930e71390257514f P a5261a972e1c4bf0bbbe47c26c34842f: Deleting consensus metadata
I20260810 19:35:10.347069 12407 catalog_manager.cc:4841] TS a5261a972e1c4bf0bbbe47c26c34842f (127.29.233.130:33633): tablet 678836230fe6473c930e71390257514f (table test-table [id=3c071ce46a0b4e73ae4acd376fb88d6c]) successfully deleted
I20260810 19:35:10.442545 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 12464
I20260810 19:35:10.464149 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 12644
W20260810 19:35:10.464126 12409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.233.129:38467 (error 108)
I20260810 19:35:10.466245 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.465711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34918)
0810 19:35:10.465912 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.466058 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 19:35:10.466565 12409 catalog_manager.cc:4584] TS 9b694642140a43ef9221282873844de9 (127.29.233.129:38467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 678836230fe6473c930e71390257514f: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
I20260810 19:35:10.478217 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.464996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.29.233.130:33633)
0810 19:35:10.465201 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.465207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.465359 (+ 152us) server_negotiation.cc:407] Connection header received
0810 19:35:10.465592 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.465598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.465686 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.465810 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:10.467460 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.468181 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.469434 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.469835 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.469934 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.472608 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:10.472623 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:10.472638 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:10.472667 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:10.475453 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.476092 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.476099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.476101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.476221 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.476617 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.476622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.476626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.476992 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:10.477323 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.477631 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.477850 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:35:10.478948 12796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:10.520449 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.519897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34922)
0810 19:35:10.520190 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.520304 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 19:35:10.521584 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.521229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34936)
0810 19:35:10.521358 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.521453 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 19:35:10.569658 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 12644
I20260810 19:35:10.582235 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.581589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34946)
0810 19:35:10.581861 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.581996 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 19:35:10.583444 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.583003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34954)
0810 19:35:10.583189 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.583286 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 19:35:10.591413 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 12807
I20260810 19:35:10.605769 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.592173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49882 (local address 127.29.233.131:36241)
0810 19:35:10.592357 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:10.592363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:10.592383 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:35:10.592577 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:10.592581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:10.592658 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:10.592763 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:10.594098 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.594926 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.596286 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:10.596706 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:10.596842 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:10.599849 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:10.599863 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:10.599881 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:10.599919 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:10.602916 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.603627 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.603635 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.603638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.603770 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:10.604180 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:10.604186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:10.604189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:10.604543 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:10.604868 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:10.605180 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 19:35:10.605421 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:35:10.606526 12952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:10.664196 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.663654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34962)
0810 19:35:10.663900 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.664011 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 19:35:10.665313 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:10.664875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:38467 (local address 127.0.0.1:34966)
0810 19:35:10.665119 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:10.665196 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:38467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 19:35:10.697256 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 12807
I20260810 19:35:10.717571 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 12388
I20260810 19:35:10.718885 12455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:10.834251 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 12388
I20260810 19:35:10.861239 30630 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 19:35:11.206632 30630 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 19:35:11.206859 30630 mini_ranger.cc:66] Stopping Ranger...
I20260810 19:35:11.552521 30630 mini_ranger.cc:68] Stopped Ranger
2026-08-10 19:35:11.552 UTC [11333] LOG: received smart shutdown request
2026-08-10 19:35:11.556 UTC [11333] LOG: background worker "logical replication launcher" (PID 11343) exited with exit code 1
2026-08-10 19:35:11.556 UTC [11337] LOG: shutting down
2026-08-10 19:35:11.737 UTC [11333] LOG: database system is shut down
2026-08-10T19:35:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53342 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:35:11 dist-test-slave-tm1g krb5kdc[13030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:35:11 dist-test-slave-tm1g krb5kdc[13030](info): set up 2 sockets
Aug 10 19:35:11 dist-test-slave-tm1g krb5kdc[13030](info): commencing operation
krb5kdc: starting...
W20260810 19:35:13.039950 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:35:13 dist-test-slave-tm1g krb5kdc[13030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:35:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:35:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:35:13.334113 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:46029
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46029,127.29.233.189:35237,127.29.233.188:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:13.675489 13045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:13.675778 13045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:13.676013 13045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:13.683960 13045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:13.684072 13045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:13.684165 13045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:13.684279 13045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:13.684348 13045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:13.694272 13045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46029,127.29.233.189:35237,127.29.233.188:43171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:46029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:13.695706 13045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:13.697037 13045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:13.709244 13053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:13.709270 13051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:13.709774 13050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:13.710390 13045 server_base.cc:1034] running on GCE node
I20260810 19:35:13.711187 13045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:13.712503 13045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:13.713713 13045 hybrid_clock.cc:648] HybridClock initialized: now 1786390513713673 us; error 31 us; skew 500 ppm
I20260810 19:35:13.714097 13045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:13 dist-test-slave-tm1g krb5kdc[13030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:13.722680 13045 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:13.724251 13045 webserver.cc:460] Webserver started at http://127.29.233.190:44901/ using document root <none> and password file <none>
I20260810 19:35:13.724988 13045 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:13.725144 13045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:13.725469 13045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:13.728600 13045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "93a21995b4004bc6809bff5a5a26b767"
format_stamp: "Formatted at 2026-08-10 19:35:13 on dist-test-slave-tm1g"
server_key: "920d4a64c972788136a3531c7b24d361"
server_key_iv: "21d3a32d79143e6bff4ac23af04b09ed"
server_key_version: "encryptionkey@0"
I20260810 19:35:13.729424 13045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "93a21995b4004bc6809bff5a5a26b767"
format_stamp: "Formatted at 2026-08-10 19:35:13 on dist-test-slave-tm1g"
server_key: "920d4a64c972788136a3531c7b24d361"
server_key_iv: "21d3a32d79143e6bff4ac23af04b09ed"
server_key_version: "encryptionkey@0"
I20260810 19:35:13.735311 13045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260810 19:35:13.738727 13061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:13.740363 13045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:35:13.740559 13045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "93a21995b4004bc6809bff5a5a26b767"
format_stamp: "Formatted at 2026-08-10 19:35:13 on dist-test-slave-tm1g"
server_key: "920d4a64c972788136a3531c7b24d361"
server_key_iv: "21d3a32d79143e6bff4ac23af04b09ed"
server_key_version: "encryptionkey@0"
I20260810 19:35:13.740842 13045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:13.775054 13045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:13.779232 13045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:13.779551 13045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:13.798799 13045 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:46029
I20260810 19:35:13.798843 13117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:46029 every 8 connection(s)
I20260810 19:35:13.800179 13045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:13.802587 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13045
I20260810 19:35:13.802848 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:35:13.803692 30630 external_mini_cluster.cc:1428] Setting key b827604ee35852ab1c897936510ef94b
I20260810 19:35:13.804714 13118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:13.815033 13118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:35:13 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.233.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.189@KRBTEST.COM" created.
I20260810 19:35:13.839684 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.820032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.29.233.190:46029)
0810 19:35:13.820784 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:13.820795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:13.820848 (+ 53us) server_negotiation.cc:407] Connection header received
0810 19:35:13.822114 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:13.822124 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:13.822472 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:13.822720 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:13.823886 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:13.825937 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:13.827428 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:13.828426 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:13.828644 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:13.832130 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:13.832142 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:13.832159 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:13.832196 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:13.835405 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:13.836234 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:13.836240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:13.836243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:13.836390 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:13.836858 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:13.836867 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:13.836873 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:13.837413 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:13.837567 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:13.838281 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 19:35:13.838634 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":507,"thread_start_us":333,"threads_started":1}
I20260810 19:35:13.839720 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.818670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46029 (local address 127.0.0.1:59370)
0810 19:35:13.819578 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:13.819627 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:13.821724 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:13.822732 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:13.822737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:13.823315 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:13.823672 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:13.823690 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:13.826098 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:13.826102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:13.827281 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:13.827294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:13.828660 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:13.828665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:13.828875 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:13.829635 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:13.829656 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:13.831884 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:13.835547 (+ 3663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:13.835558 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:13.835567 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:13.836112 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:13.836508 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:13.836516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:13.836523 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:13.836732 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:13.837574 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:13.837585 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:13.837864 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 19:35:13.838163 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 19:35:13.838846 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":118,"thread_start_us":397,"threads_started":1}
I20260810 19:35:13.842712 13118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:35:13.843756 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.843399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43756)
0810 19:35:13.843553 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:13.843654 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 19:35:13.843987 13062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
I20260810 19:35:13.845528 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.845177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43764)
0810 19:35:13.845357 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:13.845420 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 19:35:13.845715 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:35237: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab.
Entry for principal kudu/127.29.233.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab.
I20260810 19:35:13.873850 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:35237
--webserver_interface=127.29.233.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab
--principal=kudu/127.29.233.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46029,127.29.233.189:35237,127.29.233.188:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:35:13.895092 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } attempt: 1
I20260810 19:35:13.896337 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.895839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43768)
0810 19:35:13.896054 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:13.896186 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 19:35:13.897429 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.897086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43782)
0810 19:35:13.897239 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:13.897307 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 19:35:13.897603 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:35237: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
I20260810 19:35:13.965865 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } attempt: 2
I20260810 19:35:13.967185 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.966658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43796)
0810 19:35:13.966941 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:13.967032 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 19:35:13.968436 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:13.967997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43800)
0810 19:35:13.968188 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:13.968260 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 19:35:13.968628 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:35237: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
I20260810 19:35:14.059898 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } attempt: 3
I20260810 19:35:14.061085 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.060619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43814)
0810 19:35:14.060831 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.060951 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 19:35:14.062136 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.061796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43824)
0810 19:35:14.061949 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.062013 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 19:35:14.062342 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:35237: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
I20260810 19:35:14.205618 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } attempt: 4
I20260810 19:35:14.206934 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.206352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43830)
0810 19:35:14.206702 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.206795 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260810 19:35:14.208022 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.207662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43834)
0810 19:35:14.207815 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.207906 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 19:35:14.208199 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:35237: Network error: Client connection negotiation failed: client connection to 127.29.233.189:35237: connect: Connection refused (error 111)
W20260810 19:35:14.216249 13126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:14.216579 13126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:14.216738 13126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:14.224131 13126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:14.224279 13126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:14.224390 13126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:14.224505 13126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:14.224584 13126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:14.234555 13126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46029,127.29.233.189:35237,127.29.233.188:43171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab
--principal=kudu/127.29.233.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:35237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.233.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:14.235924 13126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:14.237232 13126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:14.249293 13131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:14.249722 13134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:14.249754 13132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:14.250439 13126 server_base.cc:1034] running on GCE node
I20260810 19:35:14.251197 13126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:14.252564 13126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:14.253806 13126 hybrid_clock.cc:648] HybridClock initialized: now 1786390514253767 us; error 69 us; skew 500 ppm
I20260810 19:35:14.254177 13126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:14.257313 13126 init.cc:377] Logged in from keytab as kudu/127.29.233.189@KRBTEST.COM (short username kudu)
I20260810 19:35:14.258790 13126 webserver.cc:460] Webserver started at http://127.29.233.189:40595/ using document root <none> and password file <none>
I20260810 19:35:14.259457 13126 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:14.259557 13126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:14.259807 13126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:14.262778 13126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data/instance:
uuid: "ae336c20a4b24ea78938f8795616e56f"
format_stamp: "Formatted at 2026-08-10 19:35:14 on dist-test-slave-tm1g"
server_key: "649f5c24711f969e5c312d136cbb4bd9"
server_key_iv: "cb0ce08a7ce7bccb6be1d3b240fe5073"
server_key_version: "encryptionkey@0"
I20260810 19:35:14.263542 13126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/wal/instance:
uuid: "ae336c20a4b24ea78938f8795616e56f"
format_stamp: "Formatted at 2026-08-10 19:35:14 on dist-test-slave-tm1g"
server_key: "649f5c24711f969e5c312d136cbb4bd9"
server_key_iv: "cb0ce08a7ce7bccb6be1d3b240fe5073"
server_key_version: "encryptionkey@0"
I20260810 19:35:14.268718 13126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:35:14.272030 13141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:14.273384 13126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:35:14.273572 13126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/wal
uuid: "ae336c20a4b24ea78938f8795616e56f"
format_stamp: "Formatted at 2026-08-10 19:35:14 on dist-test-slave-tm1g"
server_key: "649f5c24711f969e5c312d136cbb4bd9"
server_key_iv: "cb0ce08a7ce7bccb6be1d3b240fe5073"
server_key_version: "encryptionkey@0"
I20260810 19:35:14.273761 13126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:14.290195 13126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:14.294646 13126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:14.294970 13126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:14.315203 13126 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.189:35237
I20260810 19:35:14.315315 13197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.189:35237 every 8 connection(s)
I20260810 19:35:14.316624 13126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
I20260810 19:35:14.320359 13198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:14.321561 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13126
I20260810 19:35:14.321756 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-1/wal/instance
I20260810 19:35:14.322268 30630 external_mini_cluster.cc:1428] Setting key 4eb5760e5b35bcb4761b0739469161f3
I20260810 19:35:14.327051 13198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:14.349664 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.329771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59384 (local address 127.29.233.190:46029)
0810 19:35:14.329966 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.329971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.331209 (+ 1238us) server_negotiation.cc:407] Connection header received
0810 19:35:14.333122 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.333128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.333222 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.333349 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.334924 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.336137 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.337678 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.338501 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.338637 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.342067 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.342076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.342084 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.342136 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.346071 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.347027 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.347033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.347039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.347220 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.347676 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.347683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.347689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.348155 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.348347 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.349024 (+ 677us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.349278 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:35:14.350453 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.330203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46029 (local address 127.0.0.1:59384)
0810 19:35:14.331039 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.331084 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:14.332850 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:14.333580 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:14.333589 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:14.334087 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:14.334722 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.334740 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.336293 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.336297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.337520 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.337537 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.338730 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.338735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.338922 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.339623 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:14.339651 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:14.341810 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:14.346221 (+ 4411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.346228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.346241 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.346888 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.347340 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.347344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.347346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.347537 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.348291 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:14.348297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:14.348635 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 19:35:14.348859 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 19:35:14.349240 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for kudu/127.29.233.188@KRBTEST.COM; defaulting to no policy
I20260810 19:35:14.352420 13198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.188@KRBTEST.COM" created.
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:35:14.369448 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.353355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.29.233.189:35237)
0810 19:35:14.354094 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.354101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.354125 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:35:14.354199 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.354206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.354495 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.354661 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.355382 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.357112 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.358518 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.359480 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.359631 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.362780 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.362796 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.362816 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.362859 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.366440 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.367291 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.367298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.367300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.367416 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.367846 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.367849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.367851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.368204 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.368371 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.368802 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.369145 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":552,"thread_start_us":388,"threads_started":1}
I20260810 19:35:14.369546 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.353024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43848)
0810 19:35:14.353308 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.353326 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:14.353421 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:14.354672 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:14.354676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:14.355067 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:14.355210 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.355218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.357285 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.357290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.358257 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.358269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.359656 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.359661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.359863 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.360480 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:14.360497 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:14.362554 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:14.366587 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.366595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.366602 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.367157 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.367524 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.367532 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.367538 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.367737 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.368338 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:14.368344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:14.368515 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 19:35:14.368678 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 19:35:14.369026 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260810 19:35:14.371973 13198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:35:14.373253 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.372814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40426)
0810 19:35:14.372990 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.373106 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260810 19:35:14.373649 13145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.375295 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.374972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40434)
0810 19:35:14.375127 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.375202 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 19:35:14.375608 13198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab.
Entry for principal kudu/127.29.233.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab.
I20260810 19:35:14.390401 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:43171
--webserver_interface=127.29.233.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab
--principal=kudu/127.29.233.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46029,127.29.233.189:35237,127.29.233.188:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:35:14.424934 13198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 1
I20260810 19:35:14.426213 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.425747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40438)
0810 19:35:14.425945 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.426061 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 19:35:14.427294 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.426967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40452)
0810 19:35:14.427116 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.427187 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 19:35:14.427567 13198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.495810 13198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 2
I20260810 19:35:14.497124 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.496595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40456)
0810 19:35:14.496798 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.496892 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 19:35:14.498457 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.498000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40468)
0810 19:35:14.498204 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.498280 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 19:35:14.498704 13198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.507485 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } attempt: 5
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:35:14.521595 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.508153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43862)
0810 19:35:14.508330 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.508356 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:14.508460 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:14.508850 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:14.508853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:14.509220 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:14.509406 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.509414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.510851 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.510855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.511708 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.511719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.512695 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.512700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.512838 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.513513 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:14.513540 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:14.515812 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:14.519400 (+ 3588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.519404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.519407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.519770 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.520163 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.520166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.520168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.520295 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.520893 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:14.520896 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:14.521018 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 19:35:14.521129 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 19:35:14.521336 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 19:35:14.521790 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.508214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.29.233.189:35237)
0810 19:35:14.508389 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.508395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.508418 (+ 23us) server_negotiation.cc:407] Connection header received
0810 19:35:14.508611 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.508616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.508696 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.508806 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.509568 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.510719 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.511850 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.512546 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.512693 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.516019 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.516026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.516031 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.516070 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.519232 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.519888 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.519894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.519897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.520054 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.520400 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.520405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.520408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.520776 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.520928 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.521270 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.521489 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:35:14.522779 13118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:35:14.524255 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.523707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40474)
0810 19:35:14.523942 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.524058 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 19:35:14.525449 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.525030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40488)
0810 19:35:14.525209 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.525311 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 19:35:14.525671 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.576920 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 1
I20260810 19:35:14.578277 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.577761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40494)
0810 19:35:14.577968 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.578104 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 19:35:14.579514 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.579138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40510)
0810 19:35:14.579311 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.579399 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 19:35:14.579761 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.589931 13198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 3
I20260810 19:35:14.591063 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.590616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40522)
0810 19:35:14.590818 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.590914 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 19:35:14.592167 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.591733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40526)
0810 19:35:14.591973 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.592056 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260810 19:35:14.592417 13198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.647986 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 2
I20260810 19:35:14.649107 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.648632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40528)
0810 19:35:14.648830 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.648928 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 19:35:14.650144 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.649820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40532)
0810 19:35:14.649976 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.650042 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 19:35:14.650377 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.735611 13198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 4
I20260810 19:35:14.736799 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.736324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40542)
0810 19:35:14.736554 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.736643 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 19:35:14.737816 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.737498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40558)
0810 19:35:14.737632 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.737704 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 19:35:14.738041 13198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
W20260810 19:35:14.740013 13205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:14.740324 13205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:14.740479 13205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:14.747969 13205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:14.748107 13205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:14.748207 13205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:14.748322 13205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:14.748396 13205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:14.756587 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 3
I20260810 19:35:14.757791 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.757239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40574)
0810 19:35:14.757478 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.757600 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 19:35:14.758665 13205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46029,127.29.233.189:35237,127.29.233.188:43171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab
--principal=kudu/127.29.233.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:43171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.233.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:14.758919 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.758523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40578)
0810 19:35:14.758720 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.758793 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260810 19:35:14.759214 13118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:43171: Network error: Client connection negotiation failed: client connection to 127.29.233.188:43171: connect: Connection refused (error 111)
I20260810 19:35:14.760147 13205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:14.761466 13205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:14.774039 13210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:14.773964 13214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:14.774075 13212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:14.774932 13205 server_base.cc:1034] running on GCE node
I20260810 19:35:14.775652 13205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:14.777052 13205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:14.778352 13205 hybrid_clock.cc:648] HybridClock initialized: now 1786390514778278 us; error 32 us; skew 500 ppm
I20260810 19:35:14.778743 13205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:14.781893 13205 init.cc:377] Logged in from keytab as kudu/127.29.233.188@KRBTEST.COM (short username kudu)
I20260810 19:35:14.783475 13205 webserver.cc:460] Webserver started at http://127.29.233.188:38773/ using document root <none> and password file <none>
I20260810 19:35:14.784154 13205 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:14.784255 13205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:14.784519 13205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:14.787560 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data/instance:
uuid: "c20d928e324b43e2a226c493c3539689"
format_stamp: "Formatted at 2026-08-10 19:35:14 on dist-test-slave-tm1g"
server_key: "e8ee23aa72ce4f9dbbeab305a6b502ee"
server_key_iv: "428e4cdf7c471e179a0083bd11464bb7"
server_key_version: "encryptionkey@0"
I20260810 19:35:14.788355 13205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/wal/instance:
uuid: "c20d928e324b43e2a226c493c3539689"
format_stamp: "Formatted at 2026-08-10 19:35:14 on dist-test-slave-tm1g"
server_key: "e8ee23aa72ce4f9dbbeab305a6b502ee"
server_key_iv: "428e4cdf7c471e179a0083bd11464bb7"
server_key_version: "encryptionkey@0"
I20260810 19:35:14.793555 13205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:35:14.797564 13222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:14.799105 13205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:14.799301 13205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/wal
uuid: "c20d928e324b43e2a226c493c3539689"
format_stamp: "Formatted at 2026-08-10 19:35:14 on dist-test-slave-tm1g"
server_key: "e8ee23aa72ce4f9dbbeab305a6b502ee"
server_key_iv: "428e4cdf7c471e179a0083bd11464bb7"
server_key_version: "encryptionkey@0"
I20260810 19:35:14.799511 13205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:14.828033 13205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:14.832428 13205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:14.832752 13205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:14.851439 13205 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.188:43171
I20260810 19:35:14.851480 13276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.188:43171 every 8 connection(s)
I20260810 19:35:14.852840 13205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
I20260810 19:35:14.856601 13277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:14.859778 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13205
I20260810 19:35:14.859999 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786390305122638-30630-0/minicluster-data/master-2/wal/instance
I20260810 19:35:14.860628 30630 external_mini_cluster.cc:1428] Setting key c2c4098058e465b791c0992f8c9f28c4
I20260810 19:35:14.864502 13277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:14.885147 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.867854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.29.233.190:46029)
0810 19:35:14.869090 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.869097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.869118 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:35:14.869180 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.869185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.869271 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.869380 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.870543 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.871681 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.873567 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.874248 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.874417 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.877887 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.877893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.877898 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.877937 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.881302 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.882218 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.882224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.882227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.882418 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.882958 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.882963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.882966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.883321 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.883464 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.884311 (+ 847us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.884560 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1091,"thread_start_us":934,"threads_started":1}
I20260810 19:35:14.885928 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.862630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.29.233.188:43171)
0810 19:35:14.863383 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.863395 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.863435 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:35:14.865154 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.865175 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.865589 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.865937 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.867775 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.869777 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.871746 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.872821 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.873016 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.877612 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.877621 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.877632 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.877662 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.881354 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.882238 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.882246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.882249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.882463 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.882966 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.882970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.882973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.883597 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.883864 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.884379 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.884876 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":201,"server-negotiator.queue_time_us":546,"thread_start_us":350,"threads_started":1}
I20260810 19:35:14.885946 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.867765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46029 (local address 127.0.0.1:59400)
0810 19:35:14.868606 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.868681 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:14.868864 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:14.869434 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:14.869438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:14.870011 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:14.870349 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.870366 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.871865 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.871870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.873396 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.873407 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.874510 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.874515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.874634 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.875226 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:14.875248 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:14.877686 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:14.881481 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.881492 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.881503 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.882075 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.882603 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.882608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.882612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.882818 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.883484 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:14.883489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:14.883845 (+ 356us) client_negotiation.cc:769] Sending connection context
0810 19:35:14.884169 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 19:35:14.884692 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":297,"threads_started":1}
I20260810 19:35:14.887701 13277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:14.902034 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.888725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.29.233.189:35237)
0810 19:35:14.888929 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.888935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.888955 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:35:14.889030 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.889036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.889118 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.889227 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.890102 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.890874 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.892225 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.892631 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.892724 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.896142 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.896147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.896151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.896184 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.899432 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.900197 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.900205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.900211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.900374 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.900759 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.900764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.900767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.901105 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.901267 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.901577 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.901774 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:35:14.902073 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.888582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:35237 (local address 127.0.0.1:43874)
0810 19:35:14.888740 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.888760 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:14.888863 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:14.889279 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:14.889284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:14.889718 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:14.889943 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.889954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.891033 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.891037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.892075 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.892088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.892775 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.892781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.893001 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.893583 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:14.893600 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:14.895948 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:14.899563 (+ 3615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.899567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.899569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.900073 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.900504 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.900507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.900509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.900649 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.901217 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:14.901220 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:14.901342 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 19:35:14.901450 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 19:35:14.901716 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 19:35:14.903053 13277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:35:14.904713 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.889015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59414 (local address 127.29.233.190:46029)
0810 19:35:14.889170 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.889176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.889306 (+ 130us) server_negotiation.cc:407] Connection header received
0810 19:35:14.889451 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.889455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.889526 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.889599 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.890977 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.891662 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.893148 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.893609 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.893705 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.898129 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.898137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.898144 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.898183 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.901497 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.902504 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.902509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.902513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.902636 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.903099 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.903103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.903106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.903477 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.903740 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.904147 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.904370 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:35:14.916651 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.904093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40594)
0810 19:35:14.904365 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.904382 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:14.904492 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:14.904933 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:14.904938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:14.905283 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:14.905420 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.905427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.906499 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.906504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.907945 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.907962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.908660 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.908666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.908863 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.909388 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:14.909403 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:14.911317 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:14.914154 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.914161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.914165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.914681 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.915092 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.915099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.915103 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.915262 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.915782 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:14.915786 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:14.915961 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 19:35:14.916102 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 19:35:14.916299 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260810 19:35:14.916841 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.904127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.29.233.188:43171)
0810 19:35:14.904303 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.904309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.904466 (+ 157us) server_negotiation.cc:407] Connection header received
0810 19:35:14.904606 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.904611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.904675 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.904781 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.905566 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.906364 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.908090 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.908507 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.908646 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.911490 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.911495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.911498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.911530 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.914029 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.914798 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.914802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.914804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.914959 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.915377 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.915380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.915382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.915653 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.915761 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.916230 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.916508 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 19:35:14.922010 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.906482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.29.233.189:35237)
0810 19:35:14.906621 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.906625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.906789 (+ 164us) server_negotiation.cc:407] Connection header received
0810 19:35:14.906924 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.906927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.906976 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.907055 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.908531 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.909397 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.910718 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.911259 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.911395 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.915964 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.915969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.915972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.916000 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.919350 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.920117 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.920122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.920126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.920288 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.920660 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.920665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.920668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.921007 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.921270 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.921452 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.921690 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 19:35:14.923062 13277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: Bootstrap starting.
I20260810 19:35:14.926986 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:35:14.927039 13277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:14.928287 13277 log.cc:824] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:14.931682 13277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: No bootstrap required, opened a new log
I20260810 19:35:14.936414 13118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 4
I20260810 19:35:14.953657 13277 raft_consensus.cc:348] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
I20260810 19:35:14.954178 13277 raft_consensus.cc:374] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:14.954331 13277 raft_consensus.cc:729] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20d928e324b43e2a226c493c3539689, State: Initialized, Role: FOLLOWER
I20260810 19:35:14.955366 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.932848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.29.233.190:46029)
0810 19:35:14.933059 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.933068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.933863 (+ 795us) server_negotiation.cc:407] Connection header received
0810 19:35:14.934163 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.934172 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.934267 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.934388 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.937202 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.938592 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.941375 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.942295 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.942467 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.947889 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.947899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.947903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.947958 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.951753 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.952780 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.952786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.952790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.952951 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.953434 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.953439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.953442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.953805 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.954042 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.954686 (+ 644us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.954961 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":53}
I20260810 19:35:14.957038 13277 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
Aug 10 19:35:14 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390513, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
I20260810 19:35:14.968608 13293 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:14.969031 13293 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:35:14.970234 13277 sys_catalog.cc:564] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:14.978013 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.932955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.29.233.188:43171)
0810 19:35:14.933220 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.933229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.946847 (+ 13618us) server_negotiation.cc:407] Connection header received
0810 19:35:14.947294 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.947300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.947393 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.947469 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.949972 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.951200 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.953441 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.955198 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.955311 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.962740 (+ 7429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.962749 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.962757 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.962808 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.971245 (+ 8437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.975120 (+ 3875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.975126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.975129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.975290 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.975814 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.975822 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.975826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.976349 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.976602 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.977295 (+ 693us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.977567 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 19:35:14.979362 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.934282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.29.233.189:35237)
0810 19:35:14.938551 (+ 4269us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.938557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.947029 (+ 8472us) server_negotiation.cc:407] Connection header received
0810 19:35:14.947351 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.947359 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.947514 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.947609 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.950293 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.951563 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.966421 (+ 14858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.967123 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.967923 (+ 800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.971087 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.971094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.971099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.971146 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.974642 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.975590 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.975597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.975601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.975762 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.976203 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.976208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.976211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.976613 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.976837 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.977880 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.979018 (+ 1138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4140}
I20260810 19:35:14.981633 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.936965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40616)
0810 19:35:14.937218 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:14.937237 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:14.937340 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:14.951927 (+ 14587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:14.951935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:14.952429 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:14.952646 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.952659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.954522 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.954527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.957127 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:14.957143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.958286 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.958328 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:14.958515 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.959134 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:14.959155 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:14.961889 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:14.977672 (+ 15783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.977683 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.977690 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.978519 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.979294 (+ 775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:14.979303 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:14.979322 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:14.979532 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:14.980328 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:14.980336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:14.980588 (+ 252us) client_negotiation.cc:769] Sending connection context
0810 19:35:14.980761 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 19:35:14.981127 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":801}
I20260810 19:35:14.983021 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:14.937024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40616 (local address 127.29.233.188:43171)
0810 19:35:14.951484 (+ 14460us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:14.951490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:14.951516 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:35:14.951564 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:14.951570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:14.951644 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:14.952799 (+ 1155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:14.953070 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.954354 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.957268 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:14.958103 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:14.958810 (+ 707us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:14.962116 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:14.962122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:14.962127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:14.962164 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:14.977449 (+ 15285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.978892 (+ 1443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.978900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.978906 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.979122 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:14.979711 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:14.979719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:14.979724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:14.980139 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:14.982033 (+ 1894us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:14.982157 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 19:35:14.982407 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14318,"thread_start_us":14704,"threads_started":1}
I20260810 19:35:14.989112 13118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767: Bootstrap starting.
W20260810 19:35:14.990409 13305 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 19:35:14.990597 13305 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 19:35:14.993137 13118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:14.994474 13118 log.cc:824] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:14.987394 13205 master_runner.cc:186] Error getting master registration for 127.29.233.190:46029: OK, instance_id { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" instance_seqno: 1786390513793194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 19:35:14.998592 13118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767: No bootstrap required, opened a new log
I20260810 19:35:14.998680 13205 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 19:35:15.006835 13118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
I20260810 19:35:15.007259 13118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:15.007469 13118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93a21995b4004bc6809bff5a5a26b767, State: Initialized, Role: FOLLOWER
I20260810 19:35:15.008128 13118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
I20260810 19:35:15.009137 13310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.009593 13310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:35:15.010438 13118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 19:35:15.021518 13322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 19:35:15.021678 13322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260810 19:35:15.032538 13068 tablet.cc:2369] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:35:15.021847 13045 master_runner.cc:186] Error getting master registration for 127.29.233.189:35237: OK, instance_id { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" instance_seqno: 1786390514309553 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 19:35:15.033560 13045 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 19:35:15.037264 13198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } attempt: 5
Aug 10 19:35:15 dist-test-slave-tm1g krb5kdc[13030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390514, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
I20260810 19:35:15.055555 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:15.038023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40624 (local address 127.29.233.188:43171)
0810 19:35:15.038199 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:15.038204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:15.038339 (+ 135us) server_negotiation.cc:407] Connection header received
0810 19:35:15.038580 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:15.038586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:15.038692 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:15.038807 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:15.039831 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:15.041002 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:15.042812 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:15.043560 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:15.043687 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:15.048259 (+ 4572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:15.048266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:15.048271 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:15.048313 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:15.052317 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:15.053165 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:15.053171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:15.053174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:15.053347 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:15.053842 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:15.053847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:15.053851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:15.054217 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:15.054469 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:15.054881 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 19:35:15.055146 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 19:35:15.055554 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:15.037933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:43171 (local address 127.0.0.1:40624)
0810 19:35:15.038217 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:15.038237 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:15.038430 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:15.038866 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:15.038874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:15.039443 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:15.039662 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:15.039675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:15.041155 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:15.041161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:15.042620 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:15.042636 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:15.043897 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:15.043904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:15.044115 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:15.045062 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:15.045086 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:15.048056 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:15.052503 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:15.052509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:15.052514 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:15.053020 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:15.053493 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:15.053498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:15.053502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:15.053702 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:15.054389 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:15.054394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:15.054571 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 19:35:15.054730 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 19:35:15.055026 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260810 19:35:15.056789 13310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:35:15.057116 13310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
I20260810 19:35:15.058895 13310 leader_election.cc:288] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae336c20a4b24ea78938f8795616e56f (127.29.233.189:35237), c20d928e324b43e2a226c493c3539689 (127.29.233.188:43171)
I20260810 19:35:15.059989 13252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93a21995b4004bc6809bff5a5a26b767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20d928e324b43e2a226c493c3539689" is_pre_election: true
I20260810 19:35:15.060637 13252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93a21995b4004bc6809bff5a5a26b767 in term 0.
I20260810 19:35:15.061276 13066 leader_election.cc:302] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93a21995b4004bc6809bff5a5a26b767, c20d928e324b43e2a226c493c3539689; no voters:
I20260810 19:35:15.062531 13310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:35:15.062680 13310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:35:15.062793 13310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:15.063403 13198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f: Bootstrap starting.
I20260810 19:35:15.066260 13310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
I20260810 19:35:15.067533 13252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93a21995b4004bc6809bff5a5a26b767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20d928e324b43e2a226c493c3539689"
I20260810 19:35:15.067741 13252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:15.071182 13252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93a21995b4004bc6809bff5a5a26b767 in term 1.
I20260810 19:35:15.071807 13066 leader_election.cc:302] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93a21995b4004bc6809bff5a5a26b767, c20d928e324b43e2a226c493c3539689; no voters:
I20260810 19:35:15.074177 13198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:15.075430 13310 leader_election.cc:288] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [CANDIDATE]: Term 1 election: Requested vote from peers ae336c20a4b24ea78938f8795616e56f (127.29.233.189:35237), c20d928e324b43e2a226c493c3539689 (127.29.233.188:43171)
I20260810 19:35:15.076258 13310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:15.076680 13198 log.cc:824] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:15.076893 13310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [term 1 LEADER]: Becoming Leader. State: Replica: 93a21995b4004bc6809bff5a5a26b767, State: Running, Role: LEADER
I20260810 19:35:15.078976 13310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
W20260810 19:35:15.084973 13228 tablet.cc:2369] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:35:15.093219 13198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f: No bootstrap required, opened a new log
I20260810 19:35:15.098393 13324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93a21995b4004bc6809bff5a5a26b767. Latest consensus state: current_term: 1 leader_uuid: "93a21995b4004bc6809bff5a5a26b767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.099208 13324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:15.106889 13327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:15.108989 13327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:15.110406 13198 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
I20260810 19:35:15.112865 13198 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:15.112993 13198 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae336c20a4b24ea78938f8795616e56f, State: Initialized, Role: FOLLOWER
I20260810 19:35:15.113747 13198 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } }
I20260810 19:35:15.121812 13252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [term 1 FOLLOWER]: Refusing update from remote peer 93a21995b4004bc6809bff5a5a26b767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:15.122995 13324 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:15.126888 13329 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.127308 13329 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:35:15.131501 13198 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:15.131881 13169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:15.134483 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93a21995b4004bc6809bff5a5a26b767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae336c20a4b24ea78938f8795616e56f"
I20260810 19:35:15.135943 13171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93a21995b4004bc6809bff5a5a26b767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae336c20a4b24ea78938f8795616e56f" is_pre_election: true
I20260810 19:35:15.138340 13169 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [term 1 FOLLOWER]: Refusing update from remote peer 93a21995b4004bc6809bff5a5a26b767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:15.139184 13324 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:15.161784 13293 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93a21995b4004bc6809bff5a5a26b767. Latest consensus state: current_term: 1 leader_uuid: "93a21995b4004bc6809bff5a5a26b767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.162149 13293 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:35:15.169310 13328 mvcc.cc:205] Tried to move back new op lower bound from 7317055549923602432 to 7317055549783023616. Current Snapshot: MvccSnapshot[applied={T|T < 7317055549923602432}]
I20260810 19:35:15.169924 13310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93a21995b4004bc6809bff5a5a26b767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.170239 13310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:15.174175 13327 catalog_manager.cc:1347] Generated new cluster ID: d7c80005cf9b4ad99b91722190f22b5d
I20260810 19:35:15.174278 13327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:35:15.178288 13310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93a21995b4004bc6809bff5a5a26b767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.178602 13310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:15.178545 13329 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93a21995b4004bc6809bff5a5a26b767. Latest consensus state: current_term: 1 leader_uuid: "93a21995b4004bc6809bff5a5a26b767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.178934 13329 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:35:15.181633 13293 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93a21995b4004bc6809bff5a5a26b767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.181907 13293 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:35:15.189435 13327 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:35:15.191512 13327 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:35:15.192024 13333 mvcc.cc:205] Tried to move back new op lower bound from 7317055549923602432 to 7317055549783023616. Current Snapshot: MvccSnapshot[applied={T|T < 7317055549923602432}]
I20260810 19:35:15.203111 13329 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93a21995b4004bc6809bff5a5a26b767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93a21995b4004bc6809bff5a5a26b767" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46029 } } peers { permanent_uuid: "ae336c20a4b24ea78938f8795616e56f" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 35237 } } peers { permanent_uuid: "c20d928e324b43e2a226c493c3539689" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 43171 } } }
I20260810 19:35:15.203471 13329 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:35:15.221232 13327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93a21995b4004bc6809bff5a5a26b767: Generated new TSK 0
I20260810 19:35:15.222422 13327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:35:15.257287 13349 catalog_manager.cc:1259] Loaded cluster ID: d7c80005cf9b4ad99b91722190f22b5d
I20260810 19:35:15.257413 13349 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f: loading cluster ID for follower catalog manager: success
I20260810 19:35:15.261081 13349 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f: acquiring CA information for follower catalog manager: success
I20260810 19:35:15.263474 13349 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ae336c20a4b24ea78938f8795616e56f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 19:35:15.992700 13305 catalog_manager.cc:1259] Loaded cluster ID: d7c80005cf9b4ad99b91722190f22b5d
I20260810 19:35:15.992825 13305 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: loading cluster ID for follower catalog manager: success
I20260810 19:35:15.995354 13305 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: acquiring CA information for follower catalog manager: success
I20260810 19:35:15.997047 13305 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c20d928e324b43e2a226c493c3539689: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 19:35:16.346184 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13045
I20260810 19:35:16.347873 13113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:16.459805 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13045
W20260810 19:35:16.486459 13223 connection.cc:535] server connection from 127.0.0.1:40616 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 19:35:16.486896 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13126
I20260810 19:35:16.488018 13192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:16.595181 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13126
I20260810 19:35:16.618886 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13205
I20260810 19:35:16.619994 13272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:16.729677 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13205
2026-08-10T19:35:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4864 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:35:16 dist-test-slave-tm1g krb5kdc[13360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:35:16 dist-test-slave-tm1g krb5kdc[13360](info): set up 2 sockets
Aug 10 19:35:16 dist-test-slave-tm1g krb5kdc[13360](info): commencing operation
krb5kdc: starting...
W20260810 19:35:17.913933 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:35:18 dist-test-slave-tm1g krb5kdc[13360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:35:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:35:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:35:18.206187 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35651
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:18.546020 13376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:18.546370 13376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:18.546487 13376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:18.553635 13376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:18.553938 13376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:18.554006 13376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:18.554059 13376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:18.554105 13376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:18.564049 13376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:35651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:18.565337 13376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:18.566679 13376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:18.578651 13383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:18.578665 13381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:18.578832 13385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:18.579653 13376 server_base.cc:1034] running on GCE node
I20260810 19:35:18.580426 13376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:18.581954 13376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:18.583155 13376 hybrid_clock.cc:648] HybridClock initialized: now 1786390518583119 us; error 33 us; skew 500 ppm
I20260810 19:35:18.583534 13376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:18 dist-test-slave-tm1g krb5kdc[13360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390518, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:18.586719 13376 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:18.588234 13376 webserver.cc:460] Webserver started at http://127.29.233.190:40503/ using document root <none> and password file <none>
I20260810 19:35:18.588943 13376 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:18.589097 13376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:18.589427 13376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:18.592582 13376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "847dccb918ab45afbb0f7b9bbe1d242c"
format_stamp: "Formatted at 2026-08-10 19:35:18 on dist-test-slave-tm1g"
server_key: "ccbf9e63326477d93715080b66adac9b"
server_key_iv: "9aed5d1d602e18eab6f6d5b7c8dcc1d0"
server_key_version: "encryptionkey@0"
I20260810 19:35:18.593339 13376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "847dccb918ab45afbb0f7b9bbe1d242c"
format_stamp: "Formatted at 2026-08-10 19:35:18 on dist-test-slave-tm1g"
server_key: "ccbf9e63326477d93715080b66adac9b"
server_key_iv: "9aed5d1d602e18eab6f6d5b7c8dcc1d0"
server_key_version: "encryptionkey@0"
I20260810 19:35:18.598515 13376 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:35:18.601922 13392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:18.603266 13376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:18.603482 13376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "847dccb918ab45afbb0f7b9bbe1d242c"
format_stamp: "Formatted at 2026-08-10 19:35:18 on dist-test-slave-tm1g"
server_key: "ccbf9e63326477d93715080b66adac9b"
server_key_iv: "9aed5d1d602e18eab6f6d5b7c8dcc1d0"
server_key_version: "encryptionkey@0"
I20260810 19:35:18.603681 13376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:18.622989 13376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:18.627022 13376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:18.627313 13376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:18.646420 13376 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:35651
I20260810 19:35:18.646413 13444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:35651 every 8 connection(s)
I20260810 19:35:18.647826 13376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:18.651546 13445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:18.653164 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13376
I20260810 19:35:18.653409 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:35:18.654358 30630 external_mini_cluster.cc:1428] Setting key e695b449184e5df31d3f22214c8786b1
I20260810 19:35:18.664711 13445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c: Bootstrap starting.
I20260810 19:35:18.668120 13445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c: Neither blocks nor log segments found. Creating new log.
Aug 10 19:35:18 dist-test-slave-tm1g krb5kdc[13360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:18.669152 13445 log.cc:824] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:18.672099 13445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c: No bootstrap required, opened a new log
I20260810 19:35:18.677047 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:18.656296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.29.233.190:35651)
0810 19:35:18.657066 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:18.657081 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:18.657121 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:35:18.658795 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:18.658830 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:18.659207 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:18.659581 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:18.661100 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:18.662485 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:18.663877 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:18.664556 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:18.664757 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:18.669360 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:18.669371 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:18.669389 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:18.669441 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:18.672659 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:18.673469 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:18.673476 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:18.673484 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:18.673672 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:18.674177 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:18.674184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:18.674192 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:18.674806 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:18.675125 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:18.675640 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 19:35:18.676054 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":342,"threads_started":1}
I20260810 19:35:18.680328 13445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35651 } }
I20260810 19:35:18.680716 13445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:18.680843 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:35:18.680857 13445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847dccb918ab45afbb0f7b9bbe1d242c, State: Initialized, Role: FOLLOWER
I20260810 19:35:18.681456 13445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35651 } }
I20260810 19:35:18.681715 13445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:35:18.681854 13445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:35:18.682044 13445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:18.684561 13445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35651 } }
I20260810 19:35:18.685057 13445 leader_election.cc:302] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 847dccb918ab45afbb0f7b9bbe1d242c; no voters:
I20260810 19:35:18.685662 13445 leader_election.cc:288] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:35:18.685787 13453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:18.686551 13453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [term 1 LEADER]: Becoming Leader. State: Replica: 847dccb918ab45afbb0f7b9bbe1d242c, State: Running, Role: LEADER
I20260810 19:35:18.687104 13453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35651 } }
I20260810 19:35:18.687804 13445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:18.689857 13454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35651 } } }
I20260810 19:35:18.689869 13455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 847dccb918ab45afbb0f7b9bbe1d242c. Latest consensus state: current_term: 1 leader_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "847dccb918ab45afbb0f7b9bbe1d242c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35651 } } }
I20260810 19:35:18.690331 13454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:18.690352 13455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:18.691226 13458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:18.696609 13458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:18.705086 13458 catalog_manager.cc:1347] Generated new cluster ID: 8a9ee96c799d4e32a9c41db74cfe7bff
I20260810 19:35:18.705188 13458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:35:18.714833 13458 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:35:18.716114 13458 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:35:18.723979 13458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 847dccb918ab45afbb0f7b9bbe1d242c: Generated new TSK 0
I20260810 19:35:18.724742 13458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:35:18.741607 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13376
I20260810 19:35:18.742796 13440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:18.835031 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13376
2026-08-10T19:35:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2100 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:35:18 dist-test-slave-tm1g krb5kdc[13478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:35:18 dist-test-slave-tm1g krb5kdc[13478](info): set up 2 sockets
Aug 10 19:35:18 dist-test-slave-tm1g krb5kdc[13478](info): commencing operation
krb5kdc: starting...
W20260810 19:35:20.027462 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:35:20 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:35:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:35:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:35:20.338403 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33775
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:33775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:20.674404 13492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:20.674688 13492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:20.674891 13492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:20.682016 13492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:20.682114 13492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:20.682230 13492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:20.682353 13492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:20.682430 13492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:20.692144 13492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:33775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:20.693461 13492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:20.694831 13492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:20.706681 13497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:20.706604 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:20.706732 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:20.711927 13492 server_base.cc:1034] running on GCE node
I20260810 19:35:20.712723 13492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:20.714141 13492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:20.715407 13492 hybrid_clock.cc:648] HybridClock initialized: now 1786390520715358 us; error 63 us; skew 500 ppm
I20260810 19:35:20.715768 13492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:20 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390520, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:20.718937 13492 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:20.720505 13492 webserver.cc:460] Webserver started at http://127.29.233.190:41707/ using document root <none> and password file <none>
I20260810 19:35:20.721237 13492 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:20.721392 13492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:20.721704 13492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:20.724730 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "0f4defe67901483ab1d0bd22d8977012"
format_stamp: "Formatted at 2026-08-10 19:35:20 on dist-test-slave-tm1g"
server_key: "fee6a739fad2b4bec9d776a26712b9eb"
server_key_iv: "f75245f94912eb63ea4515893074e27f"
server_key_version: "encryptionkey@0"
I20260810 19:35:20.725540 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "0f4defe67901483ab1d0bd22d8977012"
format_stamp: "Formatted at 2026-08-10 19:35:20 on dist-test-slave-tm1g"
server_key: "fee6a739fad2b4bec9d776a26712b9eb"
server_key_iv: "f75245f94912eb63ea4515893074e27f"
server_key_version: "encryptionkey@0"
I20260810 19:35:20.730629 13492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:35:20.733906 13507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:20.735190 13492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:20.735390 13492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "0f4defe67901483ab1d0bd22d8977012"
format_stamp: "Formatted at 2026-08-10 19:35:20 on dist-test-slave-tm1g"
server_key: "fee6a739fad2b4bec9d776a26712b9eb"
server_key_iv: "f75245f94912eb63ea4515893074e27f"
server_key_version: "encryptionkey@0"
I20260810 19:35:20.735589 13492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:20.762655 13492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:20.766790 13492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:20.767102 13492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:20.786828 13492 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:33775
I20260810 19:35:20.786875 13560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:33775 every 8 connection(s)
I20260810 19:35:20.788219 13492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:20.791968 13561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:20.796727 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13492
I20260810 19:35:20.797003 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:35:20.797886 30630 external_mini_cluster.cc:1428] Setting key d4cc8d13d0f89e94e3fd5c884d3893c1
I20260810 19:35:20.803900 13561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Bootstrap starting.
I20260810 19:35:20.807680 13561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:20.808945 13561 log.cc:824] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:20.812493 13561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: No bootstrap required, opened a new log
Aug 10 19:35:20 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:20.820482 13561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:20.820886 13561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:20.821020 13561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f4defe67901483ab1d0bd22d8977012, State: Initialized, Role: FOLLOWER
I20260810 19:35:20.821689 13561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:20.821965 13561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:35:20.822151 13561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:35:20.822395 13561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:20.822940 13564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:20.800087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52480 (local address 127.29.233.190:33775)
0810 19:35:20.800981 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:20.800993 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:20.801036 (+ 43us) server_negotiation.cc:407] Connection header received
0810 19:35:20.802834 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:20.802857 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:20.803229 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:20.803543 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:20.805411 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:20.806888 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:20.808878 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:20.809866 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:20.810076 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:20.815065 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:20.815073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:20.815090 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:20.815133 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:20.818411 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:20.819114 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:20.819133 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:20.819139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:20.819322 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:20.819734 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:20.819739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:20.819744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:20.820271 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:20.820611 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:20.821044 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 19:35:20.821642 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":440,"threads_started":1}
I20260810 19:35:20.824920 13561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:20.825469 13561 leader_election.cc:302] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f4defe67901483ab1d0bd22d8977012; no voters:
I20260810 19:35:20.826152 13561 leader_election.cc:288] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:35:20.826269 13566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:20.826990 13566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 LEADER]: Becoming Leader. State: Replica: 0f4defe67901483ab1d0bd22d8977012, State: Running, Role: LEADER
I20260810 19:35:20.827590 13566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:20.828481 13561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:20.830930 13567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f4defe67901483ab1d0bd22d8977012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } } }
I20260810 19:35:20.830968 13569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f4defe67901483ab1d0bd22d8977012. Latest consensus state: current_term: 1 leader_uuid: "0f4defe67901483ab1d0bd22d8977012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } } }
I20260810 19:35:20.831574 13567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:20.831660 13569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:20.833498 13574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:20.839115 13574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:20.848940 13574 catalog_manager.cc:1347] Generated new cluster ID: b48b99d5f52f438d9cdb6c98cf4fb738
I20260810 19:35:20.849056 13574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:35:20.867957 13574 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:35:20.869865 13574 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:35:20.880596 13574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Generated new TSK 0
I20260810 19:35:20.881669 13574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:35:20.956192 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33775
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:21.304666 13590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:21.305001 13590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:21.305163 13590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:21.312676 13590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:21.312836 13590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:21.313019 13590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:35:21.324154 13590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:21.326371 13590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:21.328141 13590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:21.341729 13598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:21.341817 13596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:21.341848 13595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:21.342952 13590 server_base.cc:1034] running on GCE node
I20260810 19:35:21.343557 13590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:21.344506 13590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:21.345746 13590 hybrid_clock.cc:648] HybridClock initialized: now 1786390521345704 us; error 36 us; skew 500 ppm
I20260810 19:35:21.346079 13590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:21 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390521, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:21.349282 13590 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:35:21.350787 13590 webserver.cc:460] Webserver started at http://127.29.233.129:34285/ using document root <none> and password file <none>
I20260810 19:35:21.351524 13590 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:21.351683 13590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:21.352013 13590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:21.355047 13590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "c179652fe36f4749bab6ffb0be16e8e8"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "1fbe0b240f7c5e3a827f87561dd74bd8"
server_key_iv: "435eebca3c55285f459586fac3d0f57c"
server_key_version: "encryptionkey@0"
I20260810 19:35:21.355854 13590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "c179652fe36f4749bab6ffb0be16e8e8"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "1fbe0b240f7c5e3a827f87561dd74bd8"
server_key_iv: "435eebca3c55285f459586fac3d0f57c"
server_key_version: "encryptionkey@0"
I20260810 19:35:21.361253 13590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:35:21.364697 13605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:21.365962 13590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:35:21.366164 13590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "c179652fe36f4749bab6ffb0be16e8e8"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "1fbe0b240f7c5e3a827f87561dd74bd8"
server_key_iv: "435eebca3c55285f459586fac3d0f57c"
server_key_version: "encryptionkey@0"
I20260810 19:35:21.366389 13590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:21.381036 13590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:21.385070 13590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:21.385430 13590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:21.386466 13590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:21.387804 13590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:21.387876 13590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:21.388060 13590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:21.388113 13590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:21.419821 13590 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:32869
I20260810 19:35:21.419859 13725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:32869 every 8 connection(s)
I20260810 19:35:21.421022 13590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:35:21.428447 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13590
I20260810 19:35:21.428684 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:35:21.429594 30630 external_mini_cluster.cc:1428] Setting key 3594210e25567410a855ad7c37fd61f2
Aug 10 19:35:21 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390521, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:21.441550 13728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:21.423703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:47205 (local address 127.29.233.190:33775)
0810 19:35:21.424177 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:21.424183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:21.425171 (+ 988us) server_negotiation.cc:407] Connection header received
0810 19:35:21.426978 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:21.426982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:21.427050 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:21.427150 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:21.428657 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.429872 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.431532 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.431937 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.432083 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:21.435783 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:21.435788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:21.435792 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:21.435824 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:21.438412 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:21.439188 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:21.439193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:21.439195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:21.439318 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:21.439778 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:21.439782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:21.439784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:21.440052 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:21.440237 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:21.440930 (+ 693us) server_negotiation.cc:299] Negotiation successful
0810 19:35:21.441160 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":218,"threads_started":1}
I20260810 19:35:21.442550 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:21.424285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:47205)
0810 19:35:21.425001 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:21.425044 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:21.426728 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:21.427352 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:21.427360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:21.427857 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:21.428428 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:21.428444 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.430055 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.430059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:21.431359 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:21.431371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.432188 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.432196 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:21.432440 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:21.433141 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:21.433163 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:21.435568 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:21.438553 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:21.438562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:21.438578 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:21.439018 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:21.439440 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:21.439445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:21.439450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:21.439645 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:21.440186 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:21.440195 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:21.440549 (+ 354us) client_negotiation.cc:769] Sending connection context
0810 19:35:21.440807 (+ 258us) client_negotiation.cc:240] Negotiation successful
0810 19:35:21.441342 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":282,"threads_started":1}
I20260810 19:35:21.444167 13726 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33775
I20260810 19:35:21.444515 13726 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:21.445469 13726 heartbeater.cc:511] Master 127.29.233.190:33775 requested a full tablet report, sending...
I20260810 19:35:21.447753 13525 ts_manager.cc:194] Registered new tserver with Master: c179652fe36f4749bab6ffb0be16e8e8 (127.29.233.129:32869)
I20260810 19:35:21.450008 13525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:47205
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:35:21.497604 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33775
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:21.839390 13734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:21.839702 13734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:21.839881 13734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:21.847211 13734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:21.847319 13734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:21.847541 13734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:35:21.857275 13734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:21.858629 13734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:21.859990 13734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:21.873649 13740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:21.873592 13741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:21.873618 13744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:21.874645 13734 server_base.cc:1034] running on GCE node
I20260810 19:35:21.875308 13734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:21.876225 13734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:21.877425 13734 hybrid_clock.cc:648] HybridClock initialized: now 1786390521877382 us; error 35 us; skew 500 ppm
I20260810 19:35:21.877756 13734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:21 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390521, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:21.880985 13734 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:35:21.882561 13734 webserver.cc:460] Webserver started at http://127.29.233.130:39683/ using document root <none> and password file <none>
I20260810 19:35:21.883265 13734 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:21.883397 13734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:21.883728 13734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:21.886761 13734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "2e96e737fa7a4aff9655e603f737fd38"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "90f521e9af4905747a6b6f77623ef1d1"
server_key_iv: "8f1615f43350a8908dc66eaa2cc57b2f"
server_key_version: "encryptionkey@0"
I20260810 19:35:21.887552 13734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "2e96e737fa7a4aff9655e603f737fd38"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "90f521e9af4905747a6b6f77623ef1d1"
server_key_iv: "8f1615f43350a8908dc66eaa2cc57b2f"
server_key_version: "encryptionkey@0"
I20260810 19:35:21.892622 13734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 19:35:21.895957 13753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:21.897229 13734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:35:21.897397 13734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "2e96e737fa7a4aff9655e603f737fd38"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "90f521e9af4905747a6b6f77623ef1d1"
server_key_iv: "8f1615f43350a8908dc66eaa2cc57b2f"
server_key_version: "encryptionkey@0"
I20260810 19:35:21.897605 13734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:21.916599 13734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:21.920920 13734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:21.921207 13734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:21.922168 13734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:21.923660 13734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:21.923743 13734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:21.923928 13734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:21.923983 13734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:21.959481 13734 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:33253
I20260810 19:35:21.959537 13879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:33253 every 8 connection(s)
I20260810 19:35:21.960819 13734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:35:21.966755 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13734
I20260810 19:35:21.966984 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:35:21.967460 30630 external_mini_cluster.cc:1428] Setting key badf0bc385632f5e5041455d4814dbfb
Aug 10 19:35:21 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390521, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:21.982800 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:21.963590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:43101 (local address 127.29.233.190:33775)
0810 19:35:21.964079 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:21.964084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:21.964907 (+ 823us) server_negotiation.cc:407] Connection header received
0810 19:35:21.966820 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:21.966826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:21.966931 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:21.967063 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:21.968743 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.969787 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.971506 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.971933 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.972072 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:21.975538 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:21.975550 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:21.975553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:21.975607 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:21.979048 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:21.980009 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:21.980017 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:21.980023 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:21.980169 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:21.980681 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:21.980687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:21.980693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:21.981038 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:21.981170 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:21.982028 (+ 858us) server_negotiation.cc:299] Negotiation successful
0810 19:35:21.982287 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":367,"thread_start_us":260,"threads_started":1}
I20260810 19:35:21.983623 13885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:21.964003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:43101)
0810 19:35:21.964720 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:21.964767 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:21.966543 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:21.967293 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:21.967301 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:21.967855 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:21.968565 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:21.968583 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.969931 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.969937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:21.971360 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:21.971373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:21.972165 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:21.972172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:21.972418 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:21.973104 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:21.973133 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:21.975305 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:21.979207 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:21.979219 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:21.979238 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:21.979883 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:21.980282 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:21.980289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:21.980295 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:21.980560 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:21.981171 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:21.981181 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:21.981638 (+ 457us) client_negotiation.cc:769] Sending connection context
0810 19:35:21.981871 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 19:35:21.982260 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":285,"threads_started":1}
I20260810 19:35:21.985102 13881 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33775
I20260810 19:35:21.985488 13881 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:21.986505 13881 heartbeater.cc:511] Master 127.29.233.190:33775 requested a full tablet report, sending...
I20260810 19:35:21.988441 13525 ts_manager.cc:194] Registered new tserver with Master: 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253)
I20260810 19:35:21.989284 13525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:43101
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:35:22.036028 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33775
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:22.374034 13888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:22.374325 13888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:22.374429 13888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:22.381594 13888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:22.381695 13888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:22.381806 13888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:35:22.391458 13888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:22.392746 13888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:22.394100 13888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:22.407951 13895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:22.407972 13898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:22.407893 13896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:22.408833 13888 server_base.cc:1034] running on GCE node
I20260810 19:35:22.409461 13888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:22.410550 13888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:22.411779 13888 hybrid_clock.cc:648] HybridClock initialized: now 1786390522411727 us; error 66 us; skew 500 ppm
I20260810 19:35:22.412110 13888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:22 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390522, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:22.415380 13888 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:35:22.416929 13888 webserver.cc:460] Webserver started at http://127.29.233.131:40543/ using document root <none> and password file <none>
I20260810 19:35:22.417662 13888 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:22.417816 13888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:22.418141 13888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:22.421182 13888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "641dd2cc5c1241c193ff52ea25111e38"
format_stamp: "Formatted at 2026-08-10 19:35:22 on dist-test-slave-tm1g"
server_key: "84644474175adfc3c2398742e480d651"
server_key_iv: "807d4fa0bed29d28854a3e21e412235a"
server_key_version: "encryptionkey@0"
I20260810 19:35:22.422019 13888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "641dd2cc5c1241c193ff52ea25111e38"
format_stamp: "Formatted at 2026-08-10 19:35:22 on dist-test-slave-tm1g"
server_key: "84644474175adfc3c2398742e480d651"
server_key_iv: "807d4fa0bed29d28854a3e21e412235a"
server_key_version: "encryptionkey@0"
I20260810 19:35:22.427444 13888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 19:35:22.431051 13905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:22.432437 13888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:35:22.432623 13888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "641dd2cc5c1241c193ff52ea25111e38"
format_stamp: "Formatted at 2026-08-10 19:35:22 on dist-test-slave-tm1g"
server_key: "84644474175adfc3c2398742e480d651"
server_key_iv: "807d4fa0bed29d28854a3e21e412235a"
server_key_version: "encryptionkey@0"
I20260810 19:35:22.432852 13888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:22.449306 13888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:22.452976 13726 heartbeater.cc:503] Master 127.29.233.190:33775 was elected leader, sending a full tablet report...
I20260810 19:35:22.453689 13888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:22.454020 13888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:22.455248 13888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:22.456646 13888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:22.456727 13888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:22.456877 13888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:22.456944 13888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:22.491984 13888 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:45145
I20260810 19:35:22.492020 14027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:45145 every 8 connection(s)
I20260810 19:35:22.493302 13888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:35:22.495649 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 13888
I20260810 19:35:22.495867 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:35:22.496445 30630 external_mini_cluster.cc:1428] Setting key ae4e6e5e3d70f5e9e813ad68ceaafc7b
Aug 10 19:35:22 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390522, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:22.513797 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.496318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:39385 (local address 127.29.233.190:33775)
0810 19:35:22.496915 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.496922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.498386 (+ 1464us) server_negotiation.cc:407] Connection header received
0810 19:35:22.499856 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.499861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.499946 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.500069 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:22.501610 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.502512 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.504033 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.504450 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.504585 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.507965 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:22.507970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:22.507973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:22.508007 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:22.510705 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:22.511660 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:22.511666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:22.511669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:22.511855 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:22.512326 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:22.512330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:22.512332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:22.512603 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:22.512725 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.513355 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.513526 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":267,"threads_started":1}
I20260810 19:35:22.514810 14031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.496933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:39385)
0810 19:35:22.497810 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:22.497860 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:22.499598 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:22.500308 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:22.500316 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:22.500895 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:22.501444 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.501461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.502661 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.502667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.503870 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.503891 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.504672 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.504680 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.504928 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.505571 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:22.505592 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:22.507756 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:22.510902 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:22.510912 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:22.510928 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:22.511544 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:22.511976 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:22.511979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:22.511982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:22.512214 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:22.512711 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:22.512717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:22.513043 (+ 326us) client_negotiation.cc:769] Sending connection context
0810 19:35:22.513271 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 19:35:22.513639 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":396,"threads_started":1}
I20260810 19:35:22.516202 14028 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33775
I20260810 19:35:22.516505 14028 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:22.517230 14028 heartbeater.cc:511] Master 127.29.233.190:33775 requested a full tablet report, sending...
I20260810 19:35:22.518636 13525 ts_manager.cc:194] Registered new tserver with Master: 641dd2cc5c1241c193ff52ea25111e38 (127.29.233.131:45145)
I20260810 19:35:22.519409 13525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:39385
I20260810 19:35:22.524401 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:35:22.542804 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.529112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.29.233.190:33775)
0810 19:35:22.529291 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.529296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.529836 (+ 540us) server_negotiation.cc:407] Connection header received
0810 19:35:22.530021 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.530025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.530134 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.530215 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:22.531717 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.532456 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.533813 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.534209 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.534350 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.537175 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:22.537180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:22.537183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:22.537211 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:22.540162 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:22.540924 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:22.540928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:22.540930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:22.541050 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:22.541434 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:22.541438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:22.541441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:22.541798 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:22.541996 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.542396 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.542576 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:35:22.547092 13525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:35:22.553978 13525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:35:22.581061 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.567681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.29.233.130:33253)
0810 19:35:22.570571 (+ 2890us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.570579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.572313 (+ 1734us) server_negotiation.cc:407] Connection header received
0810 19:35:22.572482 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.572490 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.572711 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.572914 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:22.573760 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.575604 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.578251 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.579811 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.579971 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.580152 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.580508 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.580691 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2707,"thread_start_us":1777,"threads_started":1}
I20260810 19:35:22.581156 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.567079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33253 (local address 127.0.0.1:54064)
0810 19:35:22.572190 (+ 5111us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:22.572204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:22.572346 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:22.572926 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:22.572933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:22.572969 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:22.573522 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.573538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.575805 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.575811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.578051 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.578066 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.580023 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.580031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.580237 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.580292 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 19:35:22.580439 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 19:35:22.580624 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5015,"mutex_wait_us":25,"thread_start_us":2023,"threads_started":1}
I20260810 19:35:22.581288 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.566792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45145 (local address 127.0.0.1:60892)
0810 19:35:22.570065 (+ 3273us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:22.570105 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:22.570321 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:22.571284 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:22.571295 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:22.571341 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:22.572794 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.572823 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.574721 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.574729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.578186 (+ 3457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.578201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.580349 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.580353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.580533 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.580562 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 19:35:22.580648 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 19:35:22.580771 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2985,"mutex_wait_us":93,"thread_start_us":5146,"threads_started":1}
I20260810 19:35:22.582129 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.567084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.29.233.131:45145)
0810 19:35:22.568166 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.568179 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.570659 (+ 2480us) server_negotiation.cc:407] Connection header received
0810 19:35:22.570772 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.570780 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.571033 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.571229 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:22.573042 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.574587 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.578393 (+ 3806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.579800 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.579976 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.580152 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.581688 (+ 1536us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.581862 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":635,"threads_started":1}
I20260810 19:35:22.584086 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.570573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49552)
0810 19:35:22.575382 (+ 4809us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:22.575405 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:22.575526 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:22.577189 (+ 1663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:22.577194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:22.577219 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:22.577658 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.577668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.579314 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.579320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.581179 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.581196 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.583295 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.583302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.583487 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.583518 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 19:35:22.583620 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 19:35:22.583759 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4645,"thread_start_us":4313,"threads_started":1}
I20260810 19:35:22.584198 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.569475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.29.233.129:32869)
0810 19:35:22.570195 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.570206 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.576725 (+ 6519us) server_negotiation.cc:407] Connection header received
0810 19:35:22.576846 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.576852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.577010 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.577198 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:22.577845 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.579165 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.581399 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.582744 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.582955 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.583145 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.583683 (+ 538us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.583865 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":1201,"threads_started":1}
I20260810 19:35:22.585683 13806 tablet_service.cc:1467] Processing CreateTablet for tablet dec8f19405664cd3b126f33a62b35db2 (DEFAULT_TABLE table=test-table [id=bcb5326eac714111ba756a53f3e320f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:22.585865 13958 tablet_service.cc:1467] Processing CreateTablet for tablet dec8f19405664cd3b126f33a62b35db2 (DEFAULT_TABLE table=test-table [id=bcb5326eac714111ba756a53f3e320f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:22.586987 13806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec8f19405664cd3b126f33a62b35db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:22.587001 13958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec8f19405664cd3b126f33a62b35db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:22.592198 13658 tablet_service.cc:1467] Processing CreateTablet for tablet dec8f19405664cd3b126f33a62b35db2 (DEFAULT_TABLE table=test-table [id=bcb5326eac714111ba756a53f3e320f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:22.593626 13658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec8f19405664cd3b126f33a62b35db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:22.598377 14052 tablet_bootstrap.cc:492] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Bootstrap starting.
I20260810 19:35:22.601271 14054 tablet_bootstrap.cc:492] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Bootstrap starting.
I20260810 19:35:22.602748 14052 tablet_bootstrap.cc:654] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:22.604064 14052 log.cc:824] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:22.605343 14054 tablet_bootstrap.cc:654] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:22.606887 14054 log.cc:824] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:22.607070 14056 tablet_bootstrap.cc:492] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Bootstrap starting.
I20260810 19:35:22.610440 14056 tablet_bootstrap.cc:654] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:22.610445 14054 tablet_bootstrap.cc:492] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: No bootstrap required, opened a new log
I20260810 19:35:22.610740 14054 ts_tablet_manager.cc:1397] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 19:35:22.611229 14052 tablet_bootstrap.cc:492] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: No bootstrap required, opened a new log
I20260810 19:35:22.611516 14052 ts_tablet_manager.cc:1397] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260810 19:35:22.611506 14056 log.cc:824] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:22.613828 14056 tablet_bootstrap.cc:492] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: No bootstrap required, opened a new log
I20260810 19:35:22.614080 14056 ts_tablet_manager.cc:1397] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 19:35:22.621518 14056 raft_consensus.cc:348] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.621554 14054 raft_consensus.cc:348] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.621894 14056 raft_consensus.cc:374] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:22.622010 14056 raft_consensus.cc:729] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c179652fe36f4749bab6ffb0be16e8e8, State: Initialized, Role: FOLLOWER
I20260810 19:35:22.622069 14054 raft_consensus.cc:374] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:22.622219 14054 raft_consensus.cc:729] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e96e737fa7a4aff9655e603f737fd38, State: Initialized, Role: FOLLOWER
I20260810 19:35:22.622083 14052 raft_consensus.cc:348] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.622567 14052 raft_consensus.cc:374] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:22.622682 14052 raft_consensus.cc:729] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641dd2cc5c1241c193ff52ea25111e38, State: Initialized, Role: FOLLOWER
I20260810 19:35:22.622707 14056 consensus_queue.cc:260] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.622973 14054 consensus_queue.cc:260] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.623215 14052 consensus_queue.cc:260] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.624671 14056 ts_tablet_manager.cc:1428] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
I20260810 19:35:22.627110 13881 heartbeater.cc:503] Master 127.29.233.190:33775 was elected leader, sending a full tablet report...
I20260810 19:35:22.629742 14054 ts_tablet_manager.cc:1428] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
I20260810 19:35:22.630548 14028 heartbeater.cc:503] Master 127.29.233.190:33775 was elected leader, sending a full tablet report...
I20260810 19:35:22.631299 14052 ts_tablet_manager.cc:1428] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Time spent starting tablet: real 0.020s user 0.015s sys 0.005s
W20260810 19:35:22.673324 13727 tablet.cc:2369] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:35:22.713011 13882 tablet.cc:2369] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:35:22.745258 14029 tablet.cc:2369] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:35:22.798599 14059 raft_consensus.cc:482] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:35:22.798943 14059 raft_consensus.cc:504] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.800894 14059 leader_election.cc:288] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641dd2cc5c1241c193ff52ea25111e38 (127.29.233.131:45145), 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253)
I20260810 19:35:22.808068 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.801399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:33267 (local address 127.29.233.131:45145)
0810 19:35:22.801687 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.801696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.802253 (+ 557us) server_negotiation.cc:407] Connection header received
0810 19:35:22.802632 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.802640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.802724 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.802807 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:22.803705 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.804455 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.806365 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.807120 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.807307 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.807396 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.807687 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.807811 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 19:35:22.808182 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.801520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:40093 (local address 127.29.233.130:33253)
0810 19:35:22.801743 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.801751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.802419 (+ 668us) server_negotiation.cc:407] Connection header received
0810 19:35:22.802726 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.802737 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.802855 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.802949 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:22.803671 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.804426 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.806453 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.807218 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.807396 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.807478 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.807787 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.807922 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 19:35:22.808256 14063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.801237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45145 (local address 127.29.233.129:33267)
0810 19:35:22.802138 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:22.802159 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:22.802432 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:22.803033 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:22.803037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:22.803065 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:22.803547 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.803558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.804596 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.804601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.806181 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.806193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.807340 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.807346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.807520 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.807547 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 19:35:22.807660 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 19:35:22.807788 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":1,"thread_start_us":1080,"threads_started":1}
I20260810 19:35:22.808284 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.801244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33253 (local address 127.29.233.129:40093)
0810 19:35:22.802270 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:22.802285 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:22.802549 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:22.802996 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:22.803001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:22.803029 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:22.803499 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.803514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.804561 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.804569 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.806236 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:22.806248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.807449 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.807455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:22.807626 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.807648 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 19:35:22.807754 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 19:35:22.807878 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":33,"thread_start_us":735,"threads_started":1}
I20260810 19:35:22.809297 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec8f19405664cd3b126f33a62b35db2" candidate_uuid: "c179652fe36f4749bab6ffb0be16e8e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e96e737fa7a4aff9655e603f737fd38" is_pre_election: true
I20260810 19:35:22.809692 13978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec8f19405664cd3b126f33a62b35db2" candidate_uuid: "c179652fe36f4749bab6ffb0be16e8e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641dd2cc5c1241c193ff52ea25111e38" is_pre_election: true
I20260810 19:35:22.809942 13829 raft_consensus.cc:2454] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c179652fe36f4749bab6ffb0be16e8e8 in term 0.
I20260810 19:35:22.810209 13978 raft_consensus.cc:2454] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c179652fe36f4749bab6ffb0be16e8e8 in term 0.
I20260810 19:35:22.810775 13607 leader_election.cc:302] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e96e737fa7a4aff9655e603f737fd38, c179652fe36f4749bab6ffb0be16e8e8; no voters:
I20260810 19:35:22.811236 14059 raft_consensus.cc:2781] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:35:22.811437 14059 raft_consensus.cc:482] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:35:22.811553 14059 raft_consensus.cc:3037] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:22.813723 14059 raft_consensus.cc:504] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.814327 14059 leader_election.cc:288] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [CANDIDATE]: Term 1 election: Requested vote from peers 641dd2cc5c1241c193ff52ea25111e38 (127.29.233.131:45145), 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253)
I20260810 19:35:22.814930 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec8f19405664cd3b126f33a62b35db2" candidate_uuid: "c179652fe36f4749bab6ffb0be16e8e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e96e737fa7a4aff9655e603f737fd38"
I20260810 19:35:22.814922 13978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec8f19405664cd3b126f33a62b35db2" candidate_uuid: "c179652fe36f4749bab6ffb0be16e8e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641dd2cc5c1241c193ff52ea25111e38"
I20260810 19:35:22.815150 13829 raft_consensus.cc:3037] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:22.815150 13978 raft_consensus.cc:3037] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:22.818506 13829 raft_consensus.cc:2454] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c179652fe36f4749bab6ffb0be16e8e8 in term 1.
I20260810 19:35:22.818507 13978 raft_consensus.cc:2454] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c179652fe36f4749bab6ffb0be16e8e8 in term 1.
I20260810 19:35:22.818996 13607 leader_election.cc:302] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e96e737fa7a4aff9655e603f737fd38, c179652fe36f4749bab6ffb0be16e8e8; no voters:
I20260810 19:35:22.819303 14059 raft_consensus.cc:2781] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:22.819783 14059 raft_consensus.cc:686] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 LEADER]: Becoming Leader. State: Replica: c179652fe36f4749bab6ffb0be16e8e8, State: Running, Role: LEADER
I20260810 19:35:22.820288 14059 consensus_queue.cc:237] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } }
I20260810 19:35:22.824887 13525 catalog_manager.cc:5495] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c179652fe36f4749bab6ffb0be16e8e8 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "c179652fe36f4749bab6ffb0be16e8e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } health_report { overall_health: HEALTHY } } }
I20260810 19:35:22.873169 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.867791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.29.233.129:32869)
0810 19:35:22.867989 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.867994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.868014 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:35:22.868173 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.868176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.868276 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.868404 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:35:22.868770 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.869525 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.871146 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.871584 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.871686 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.872149 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:35:22.872686 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:35:22.872797 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.872896 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.872965 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 19:35:22.881883 13829 raft_consensus.cc:1261] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Refusing update from remote peer c179652fe36f4749bab6ffb0be16e8e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:22.881876 13978 raft_consensus.cc:1261] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 FOLLOWER]: Refusing update from remote peer c179652fe36f4749bab6ffb0be16e8e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:22.883129 14059 consensus_queue.cc:1035] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:22.883496 14064 consensus_queue.cc:1035] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:22.896663 14068 mvcc.cc:205] Tried to move back new op lower bound from 7317055581713444864 to 7317055581475512320. Current Snapshot: MvccSnapshot[applied={T|T < 7317055581713444864}]
I20260810 19:35:22.909503 13525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:22.909783 13525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:22.913617 13525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Sending DeleteTablet for 3 replicas of tablet dec8f19405664cd3b126f33a62b35db2
I20260810 19:35:22.914772 13958 tablet_service.cc:1514] Processing DeleteTablet for tablet dec8f19405664cd3b126f33a62b35db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:22 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:60892
I20260810 19:35:22.914894 13658 tablet_service.cc:1514] Processing DeleteTablet for tablet dec8f19405664cd3b126f33a62b35db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:22 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:49552
I20260810 19:35:22.914884 13806 tablet_service.cc:1514] Processing DeleteTablet for tablet dec8f19405664cd3b126f33a62b35db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:22 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:54064
I20260810 19:35:22.916364 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13590
I20260810 19:35:22.934731 14078 tablet_replica.cc:330] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: stopping tablet replica
I20260810 19:35:22.935189 14078 raft_consensus.cc:2229] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:22.935703 14078 raft_consensus.cc:2258] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:22.936462 14077 tablet_replica.cc:330] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: stopping tablet replica
I20260810 19:35:22.936878 14077 raft_consensus.cc:2229] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:22.937353 14077 raft_consensus.cc:2258] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:22.939203 14079 tablet_replica.cc:330] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: stopping tablet replica
I20260810 19:35:22.939666 14079 raft_consensus.cc:2229] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:35:22.939765 14077 ts_tablet_manager.cc:1905] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:22.939765 14078 ts_tablet_manager.cc:1905] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 19:35:22 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:35:22.940467 14079 raft_consensus.cc:2258] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:22.942741 14079 ts_tablet_manager.cc:1905] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:22.947005 14078 ts_tablet_manager.cc:1918] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:22.947242 14078 log.cc:1196] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/dec8f19405664cd3b126f33a62b35db2
I20260810 19:35:22.947433 14077 ts_tablet_manager.cc:1918] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:22.947681 14077 log.cc:1196] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/dec8f19405664cd3b126f33a62b35db2
I20260810 19:35:22.947883 14078 ts_tablet_manager.cc:1939] T dec8f19405664cd3b126f33a62b35db2 P 2e96e737fa7a4aff9655e603f737fd38: Deleting consensus metadata
I20260810 19:35:22.948246 14077 ts_tablet_manager.cc:1939] T dec8f19405664cd3b126f33a62b35db2 P 641dd2cc5c1241c193ff52ea25111e38: Deleting consensus metadata
I20260810 19:35:22.949898 13508 catalog_manager.cc:4841] TS 641dd2cc5c1241c193ff52ea25111e38 (127.29.233.131:45145): tablet dec8f19405664cd3b126f33a62b35db2 (table test-table [id=bcb5326eac714111ba756a53f3e320f3]) successfully deleted
I20260810 19:35:22.950031 13509 catalog_manager.cc:4841] TS 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253): tablet dec8f19405664cd3b126f33a62b35db2 (table test-table [id=bcb5326eac714111ba756a53f3e320f3]) successfully deleted
I20260810 19:35:22.950702 14079 ts_tablet_manager.cc:1918] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:22.951344 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:22.917820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.29.233.129:32869)
0810 19:35:22.918073 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:22.918082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:22.924324 (+ 6242us) server_negotiation.cc:407] Connection header received
0810 19:35:22.925278 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:22.925284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:22.925397 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:22.925485 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:22.929617 (+ 4132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.930906 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.934061 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:22.934830 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:22.934957 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:22.941318 (+ 6361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:22.941342 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:22.941361 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:22.941413 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:22.945431 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:22.946389 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:22.946398 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:22.946402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:22.946576 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:22.947090 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:22.947097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:22.947105 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:22.947671 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:22.948021 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:22.950599 (+ 2578us) server_negotiation.cc:299] Negotiation successful
0810 19:35:22.950863 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 19:35:22.950953 14079 log.cc:1196] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/dec8f19405664cd3b126f33a62b35db2
I20260810 19:35:22.952068 14079 ts_tablet_manager.cc:1939] T dec8f19405664cd3b126f33a62b35db2 P c179652fe36f4749bab6ffb0be16e8e8: Deleting consensus metadata
I20260810 19:35:22.952234 13721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:23.054754 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13590
I20260810 19:35:23.056588 13510 catalog_manager.cc:4841] TS c179652fe36f4749bab6ffb0be16e8e8 (127.29.233.129:32869): tablet dec8f19405664cd3b126f33a62b35db2 (table test-table [id=bcb5326eac714111ba756a53f3e320f3]) successfully deleted
I20260810 19:35:23.074187 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13734
Aug 10 19:35:23 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:35:23.091017 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:23.075027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.29.233.130:33253)
0810 19:35:23.075217 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:23.075223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:23.075322 (+ 99us) server_negotiation.cc:407] Connection header received
0810 19:35:23.075544 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:23.075554 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:23.075621 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:23.075732 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:23.077062 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:23.077856 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:23.079396 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:23.079780 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:23.079923 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:23.084878 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:23.084893 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:23.084911 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:23.084953 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:23.087977 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:23.088653 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:23.088660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:23.088662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:23.088796 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:23.089212 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:23.089216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:23.089218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:23.089682 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:23.089917 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:23.090243 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 19:35:23.090532 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:35:23.091899 13875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:23.181819 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13734
I20260810 19:35:23.201906 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13888
Aug 10 19:35:23 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:35:23.217813 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:23.202773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.29.233.131:45145)
0810 19:35:23.202970 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:23.202974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:23.203000 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:35:23.203208 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:23.203213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:23.203277 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:23.203397 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:23.204722 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:23.205519 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:23.207039 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:23.207461 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:23.207561 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:23.212130 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:23.212145 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:23.212162 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:23.212201 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:23.215246 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:23.215942 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:23.215949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:23.215951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:23.216102 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:23.216436 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:23.216439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:23.216441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:23.216767 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:23.216996 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:23.217262 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 19:35:23.217429 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:35:23.218550 14022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:23.309881 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13888
I20260810 19:35:23.329926 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 13492
I20260810 19:35:23.331233 13556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:23.444031 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 13492
I20260810 19:35:23.469894 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33775
--webserver_interface=127.29.233.190
--webserver_port=41707
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:33775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:23.813834 14086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:23.814147 14086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:23.814241 14086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:23.821588 14086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:23.821700 14086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:23.821756 14086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:23.821813 14086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:23.821858 14086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:23.832319 14086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:33775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=41707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:23.833602 14086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:23.834968 14086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:23.860816 14095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:23.860713 14094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:23.860852 14097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:23.861550 14086 server_base.cc:1034] running on GCE node
I20260810 19:35:23.862485 14086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:23.864073 14086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:23.865327 14086 hybrid_clock.cc:648] HybridClock initialized: now 1786390523865276 us; error 56 us; skew 500 ppm
I20260810 19:35:23.865749 14086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:23 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390523, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:23.869413 14086 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:23.871243 14086 webserver.cc:460] Webserver started at http://127.29.233.190:41707/ using document root <none> and password file <none>
I20260810 19:35:23.872069 14086 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:23.872242 14086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:23.878504 14086 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 19:35:23.881558 14105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:23.882982 14086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:23.883167 14086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "0f4defe67901483ab1d0bd22d8977012"
format_stamp: "Formatted at 2026-08-10 19:35:20 on dist-test-slave-tm1g"
server_key: "fee6a739fad2b4bec9d776a26712b9eb"
server_key_iv: "f75245f94912eb63ea4515893074e27f"
server_key_version: "encryptionkey@0"
I20260810 19:35:23.884248 14086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:23.896569 14086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:23.900712 14086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:23.900995 14086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:23.921079 14086 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:33775
I20260810 19:35:23.921097 14158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:33775 every 8 connection(s)
I20260810 19:35:23.922770 14086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:23.928472 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 14086
I20260810 19:35:23.929291 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:32869
--local_ip_for_outbound_sockets=127.29.233.129
--tserver_master_addrs=127.29.233.190:33775
--webserver_port=34285
--webserver_interface=127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:35:23.930111 14159 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:35:23.933480 14159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Bootstrap starting.
I20260810 19:35:23.953770 14159 log.cc:824] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:23.966049 14159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:35:23.966763 14159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Bootstrap complete.
I20260810 19:35:23.976249 14159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:23.977116 14159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f4defe67901483ab1d0bd22d8977012, State: Initialized, Role: FOLLOWER
I20260810 19:35:23.977731 14159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:23.977988 14159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:35:23.978127 14159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:35:23.978358 14159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 1 FOLLOWER]: Advancing to term 2
I20260810 19:35:23.982128 14159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:23.982672 14159 leader_election.cc:302] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f4defe67901483ab1d0bd22d8977012; no voters:
I20260810 19:35:23.983326 14159 leader_election.cc:288] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 19:35:23.983486 14163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 19:35:23.984947 14163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 LEADER]: Becoming Leader. State: Replica: 0f4defe67901483ab1d0bd22d8977012, State: Running, Role: LEADER
I20260810 19:35:23.985589 14163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:23.985829 14159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:23.988840 14165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f4defe67901483ab1d0bd22d8977012. Latest consensus state: current_term: 2 leader_uuid: "0f4defe67901483ab1d0bd22d8977012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } } }
I20260810 19:35:23.989223 14165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:23.989853 14164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0f4defe67901483ab1d0bd22d8977012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } } }
I20260810 19:35:23.990151 14164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:23.998760 14173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:24.007949 14173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:24.009387 14173 catalog_manager.cc:1259] Loaded cluster ID: b48b99d5f52f438d9cdb6c98cf4fb738
I20260810 19:35:24.009510 14173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:35:24.013336 14173 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 19:35:24.031450 14173 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 19:35:24.048034 14173 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:35:24.049973 14173 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Loaded TSK: 0
I20260810 19:35:24.051578 14173 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 19:35:24.301545 14161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:24.301862 14161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:24.302032 14161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:24.309283 14161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:24.309473 14161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:24.309648 14161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:35:24.319936 14161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:32869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=34285
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:24.321283 14161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:24.322705 14161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:24.338645 14190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:24.339222 14191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:24.340402 14161 server_base.cc:1034] running on GCE node
W20260810 19:35:24.340835 14193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:24.341531 14161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:24.342612 14161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:24.343863 14161 hybrid_clock.cc:648] HybridClock initialized: now 1786390524343807 us; error 41 us; skew 500 ppm
I20260810 19:35:24.344224 14161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:24 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390524, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:24.353863 14161 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:35:24.355639 14161 webserver.cc:460] Webserver started at http://127.29.233.129:34285/ using document root <none> and password file <none>
I20260810 19:35:24.356424 14161 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:24.356573 14161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:24.362370 14161 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 19:35:24.365223 14200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:24.366793 14161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:24.366961 14161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "c179652fe36f4749bab6ffb0be16e8e8"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "1fbe0b240f7c5e3a827f87561dd74bd8"
server_key_iv: "435eebca3c55285f459586fac3d0f57c"
server_key_version: "encryptionkey@0"
I20260810 19:35:24.367952 14161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:24.395527 14161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:24.399438 14161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:24.399755 14161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:24.400787 14161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:24.402130 14161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:24.402226 14161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:24.402381 14161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:24.402496 14161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:24.437062 14161 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:32869
I20260810 19:35:24.437119 14327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:32869 every 8 connection(s)
I20260810 19:35:24.438599 14161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:35:24.440722 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 14161
I20260810 19:35:24.441576 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:33253
--local_ip_for_outbound_sockets=127.29.233.130
--tserver_master_addrs=127.29.233.190:33775
--webserver_port=39683
--webserver_interface=127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:35:24 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390524, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:24.463450 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:24.442228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:38607)
0810 19:35:24.443089 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:24.443127 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:24.444856 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:24.446281 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:24.446295 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:24.446957 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:24.447485 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:24.447502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.449134 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.449146 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:24.450442 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:24.450454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.451496 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.451504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:24.451788 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:24.452550 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:24.452587 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:24.455520 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:24.459031 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:24.459040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:24.459055 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:24.459488 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:24.459935 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:24.459941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:24.459947 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:24.460144 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:24.460887 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:24.460895 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:24.461226 (+ 331us) client_negotiation.cc:769] Sending connection context
0810 19:35:24.461498 (+ 272us) client_negotiation.cc:240] Negotiation successful
0810 19:35:24.462062 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":340,"threads_started":1}
I20260810 19:35:24.463443 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:24.441955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:38607 (local address 127.29.233.190:33775)
0810 19:35:24.442745 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:24.442763 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:24.443272 (+ 509us) server_negotiation.cc:407] Connection header received
0810 19:35:24.445442 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:24.445454 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:24.445779 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:24.446135 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:24.447726 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.448974 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.450604 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.451242 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.451455 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:24.455808 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:24.455815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:24.455828 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:24.455866 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:24.458868 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:24.459642 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:24.459648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:24.459653 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:24.459814 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:24.460274 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:24.460279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:24.460281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:24.460735 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:24.460964 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:24.461718 (+ 754us) server_negotiation.cc:299] Negotiation successful
0810 19:35:24.462112 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":350,"threads_started":1}
I20260810 19:35:24.466658 14328 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33775
I20260810 19:35:24.467042 14328 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:24.468024 14328 heartbeater.cc:511] Master 127.29.233.190:33775 requested a full tablet report, sending...
I20260810 19:35:24.470738 14122 ts_manager.cc:194] Registered new tserver with Master: c179652fe36f4749bab6ffb0be16e8e8 (127.29.233.129:32869)
I20260810 19:35:24.473121 14122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:38607
W20260810 19:35:24.783252 14332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:24.783529 14332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:24.783624 14332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:24.791188 14332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:24.791294 14332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:24.791411 14332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:35:24.800827 14332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:33253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=39683
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:24.802019 14332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:24.803313 14332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:24.816761 14337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:24.816792 14340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:24.816845 14338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:24.817497 14332 server_base.cc:1034] running on GCE node
I20260810 19:35:24.818187 14332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:24.819187 14332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:24.820429 14332 hybrid_clock.cc:648] HybridClock initialized: now 1786390524820362 us; error 64 us; skew 500 ppm
I20260810 19:35:24.820751 14332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:24 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390524, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:24.823926 14332 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:35:24.825352 14332 webserver.cc:460] Webserver started at http://127.29.233.130:39683/ using document root <none> and password file <none>
I20260810 19:35:24.826108 14332 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:24.826225 14332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:24.831668 14332 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:35:24.834400 14347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:24.835875 14332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:24.836026 14332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "2e96e737fa7a4aff9655e603f737fd38"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "90f521e9af4905747a6b6f77623ef1d1"
server_key_iv: "8f1615f43350a8908dc66eaa2cc57b2f"
server_key_version: "encryptionkey@0"
I20260810 19:35:24.837081 14332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:24.857507 14332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:24.861821 14332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:24.862116 14332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:24.863133 14332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:24.864459 14332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:24.864590 14332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:24.864704 14332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:24.864766 14332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:24.899137 14332 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:33253
I20260810 19:35:24.899230 14479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:33253 every 8 connection(s)
I20260810 19:35:24.900624 14332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:35:24.902484 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 14332
I20260810 19:35:24.903191 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:45145
--local_ip_for_outbound_sockets=127.29.233.131
--tserver_master_addrs=127.29.233.190:33775
--webserver_port=40543
--webserver_interface=127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:35:24 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390524, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:24.925132 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:24.903286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:53399 (local address 127.29.233.190:33775)
0810 19:35:24.903529 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:24.903537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:24.904798 (+ 1261us) server_negotiation.cc:407] Connection header received
0810 19:35:24.907732 (+ 2934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:24.907737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:24.907807 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:24.907902 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:24.909960 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.910736 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.913189 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.913713 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.913846 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:24.917501 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:24.917509 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:24.917517 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:24.917562 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:24.921337 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:24.922251 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:24.922259 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:24.922265 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:24.922445 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:24.922945 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:24.922953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:24.922959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:24.923389 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:24.923532 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:24.924385 (+ 853us) server_negotiation.cc:299] Negotiation successful
0810 19:35:24.924655 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 19:35:24.926124 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:24.903770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:53399)
0810 19:35:24.904626 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:24.904678 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:24.907429 (+ 2751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:24.908159 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:24.908173 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:24.908947 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:24.909773 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:24.909800 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.910887 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.910895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:24.913042 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:24.913060 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:24.913952 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:24.913961 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:24.914281 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:24.915026 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:24.915061 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:24.917292 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:24.921503 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:24.921517 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:24.921531 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:24.922117 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:24.922578 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:24.922584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:24.922588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:24.922799 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:24.923546 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:24.923553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:24.923942 (+ 389us) client_negotiation.cc:769] Sending connection context
0810 19:35:24.924237 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 19:35:24.924735 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":323,"threads_started":1}
I20260810 19:35:24.927835 14480 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33775
I20260810 19:35:24.928164 14480 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:24.928977 14480 heartbeater.cc:511] Master 127.29.233.190:33775 requested a full tablet report, sending...
I20260810 19:35:24.930692 14122 ts_manager.cc:194] Registered new tserver with Master: 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253)
I20260810 19:35:24.932207 14122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:53399
W20260810 19:35:25.246695 14484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:25.246979 14484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:25.247071 14484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:25.254073 14484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:25.254170 14484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:25.254281 14484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:35:25.263938 14484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:45145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=40543
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:25.265349 14484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:25.266763 14484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:25.280277 14489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:25.280328 14492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:25.280357 14490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:25.281409 14484 server_base.cc:1034] running on GCE node
I20260810 19:35:25.281975 14484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:25.282948 14484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:25.284165 14484 hybrid_clock.cc:648] HybridClock initialized: now 1786390525284098 us; error 50 us; skew 500 ppm
I20260810 19:35:25.284513 14484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:25 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390525, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:25.287856 14484 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:35:25.289402 14484 webserver.cc:460] Webserver started at http://127.29.233.131:40543/ using document root <none> and password file <none>
I20260810 19:35:25.290192 14484 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:25.290369 14484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:25.295619 14484 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:35:25.298224 14499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:25.299530 14484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:35:25.299701 14484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "641dd2cc5c1241c193ff52ea25111e38"
format_stamp: "Formatted at 2026-08-10 19:35:22 on dist-test-slave-tm1g"
server_key: "84644474175adfc3c2398742e480d651"
server_key_iv: "807d4fa0bed29d28854a3e21e412235a"
server_key_version: "encryptionkey@0"
I20260810 19:35:25.300648 14484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:25.312857 14484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:25.316761 14484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:25.317035 14484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:25.318046 14484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:25.319411 14484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:25.319532 14484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:25.319670 14484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:25.319762 14484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:25.354120 14484 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:45145
I20260810 19:35:25.354179 14632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:45145 every 8 connection(s)
I20260810 19:35:25.355610 14484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:35:25.356818 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 14484
Aug 10 19:35:25 dist-test-slave-tm1g krb5kdc[13478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390525, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:25.373267 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.357935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.29.233.190:33775)
0810 19:35:25.358200 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.358210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.358673 (+ 463us) server_negotiation.cc:407] Connection header received
0810 19:35:25.358895 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.358904 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.359008 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.359116 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:25.360742 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.361495 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.363135 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.363744 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.363874 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.367274 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:25.367280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:25.367285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:25.367323 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:25.370190 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:25.371031 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:25.371036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:25.371041 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:25.371196 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:25.371626 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:25.371632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:25.371635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:25.372006 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:25.372228 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.372675 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.372841 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:35:25.378615 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.358681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:59839 (local address 127.29.233.190:33775)
0810 19:35:25.359184 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.359190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.360129 (+ 939us) server_negotiation.cc:407] Connection header received
0810 19:35:25.361903 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.361909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.361979 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.362085 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:25.363512 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.364522 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.367067 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.367838 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.367980 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.372102 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:25.372109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:25.372113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:25.372150 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:25.375525 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:25.376352 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:25.376357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:25.376362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:25.376482 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:25.376912 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:25.376917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:25.376920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:25.377178 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:25.377331 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.377978 (+ 647us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.378178 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":276,"threads_started":1}
I20260810 19:35:25.379508 14638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.359119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:59839)
0810 19:35:25.359960 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:25.360004 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:25.361665 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:25.362271 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:25.362279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:25.362799 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:25.363339 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.363355 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.364712 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.364717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.366882 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.366899 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.368082 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.368091 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.368407 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.369044 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:25.369103 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:25.371829 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:25.375672 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:25.375680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:25.375694 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:25.376227 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:25.376595 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:25.376600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:25.376604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:25.376792 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:25.377316 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:25.377324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:25.377627 (+ 303us) client_negotiation.cc:769] Sending connection context
0810 19:35:25.377876 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 19:35:25.378409 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":348,"threads_started":1}
I20260810 19:35:25.380787 14633 heartbeater.cc:347] Connected to a master server at 127.29.233.190:33775
I20260810 19:35:25.381059 14633 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:25.381707 14633 heartbeater.cc:511] Master 127.29.233.190:33775 requested a full tablet report, sending...
I20260810 19:35:25.383024 14122 ts_manager.cc:194] Registered new tserver with Master: 641dd2cc5c1241c193ff52ea25111e38 (127.29.233.131:45145)
I20260810 19:35:25.383805 14122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:59839
I20260810 19:35:25.388134 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:35:25.395085 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.389074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.29.233.190:33775)
0810 19:35:25.389221 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.389225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.389806 (+ 581us) server_negotiation.cc:407] Connection header received
0810 19:35:25.390005 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.390012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.390177 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.390292 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:35:25.390872 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.391610 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.393131 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.393523 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.393618 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.394054 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:35:25.394591 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:35:25.394703 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.394818 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.394878 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 19:35:25.395890 14122 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:35:25.403196 14122 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:35:25.445377 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.414798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.29.233.129:32869)
0810 19:35:25.421783 (+ 6985us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.421796 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.435289 (+ 13493us) server_negotiation.cc:407] Connection header received
0810 19:35:25.436203 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.436215 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.436504 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.436705 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:25.437369 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.439206 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.442244 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.443751 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.443943 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.444102 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.444842 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.445009 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6781,"thread_start_us":6643,"threads_started":1}
I20260810 19:35:25.445465 14651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.414758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49590)
0810 19:35:25.435133 (+ 20375us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:25.435158 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:25.435991 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:25.436731 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:25.436741 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:25.436776 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:25.437153 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.437166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.439403 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.439409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.442057 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.442073 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.444376 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.444383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.444580 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.444634 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 19:35:25.444795 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 19:35:25.444998 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20207,"mutex_wait_us":611,"thread_start_us":20544,"threads_started":1}
I20260810 19:35:25.445757 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.415183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33253 (local address 127.0.0.1:54080)
0810 19:35:25.435772 (+ 20589us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:25.435790 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:25.435920 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:25.437986 (+ 2066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:25.437993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:25.438026 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:25.438515 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.438531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.440692 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.440697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.442983 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.442998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.444947 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.444954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.445117 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.445156 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 19:35:25.445223 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 19:35:25.445383 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20458,"thread_start_us":20343,"threads_started":1}
I20260810 19:35:25.446697 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.415578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.29.233.130:33253)
0810 19:35:25.437385 (+ 21807us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.437393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.437425 (+ 32us) server_negotiation.cc:407] Connection header received
0810 19:35:25.437526 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.437536 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.437754 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.438718 (+ 964us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:25.438959 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.440528 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.443211 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.444689 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.446010 (+ 1321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.446160 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.446222 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.446402 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21625,"thread_start_us":26157,"threads_started":1}
I20260810 19:35:25.447484 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.414747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:45145 (local address 127.0.0.1:60912)
0810 19:35:25.426578 (+ 11831us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:25.426622 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:25.426809 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:25.437140 (+ 10331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:25.437144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:25.437173 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:25.437585 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.437597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.441075 (+ 3478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.441087 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.444326 (+ 3239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.444342 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.446642 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.446648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.446850 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.446891 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 19:35:25.446976 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 19:35:25.447105 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11587,"mutex_wait_us":46,"thread_start_us":11044,"threads_started":1}
I20260810 19:35:25.448498 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.415418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.29.233.131:45145)
0810 19:35:25.436535 (+ 21117us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.436547 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.436582 (+ 35us) server_negotiation.cc:407] Connection header received
0810 19:35:25.436677 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.436684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.436898 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.438943 (+ 2045us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:25.439241 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.440919 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.444552 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.446423 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.446632 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.447948 (+ 1316us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.448020 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.448162 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20913,"thread_start_us":20750,"threads_started":1}
I20260810 19:35:25.449723 14409 tablet_service.cc:1467] Processing CreateTablet for tablet 57595d7eb78c48bd91ec311dccc76c05 (DEFAULT_TABLE table=test-table [id=cd5a9e92766d4b05871f8f21cee0eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:25.449726 14254 tablet_service.cc:1467] Processing CreateTablet for tablet 57595d7eb78c48bd91ec311dccc76c05 (DEFAULT_TABLE table=test-table [id=cd5a9e92766d4b05871f8f21cee0eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:25.451213 14409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57595d7eb78c48bd91ec311dccc76c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:25.451313 14254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57595d7eb78c48bd91ec311dccc76c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:25.451951 14555 tablet_service.cc:1467] Processing CreateTablet for tablet 57595d7eb78c48bd91ec311dccc76c05 (DEFAULT_TABLE table=test-table [id=cd5a9e92766d4b05871f8f21cee0eb38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:25.453220 14555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57595d7eb78c48bd91ec311dccc76c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:25.463176 14654 tablet_bootstrap.cc:492] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Bootstrap starting.
I20260810 19:35:25.466727 14654 tablet_bootstrap.cc:654] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:25.466912 14655 tablet_bootstrap.cc:492] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Bootstrap starting.
I20260810 19:35:25.468199 14654 log.cc:824] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:25.468448 14656 tablet_bootstrap.cc:492] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Bootstrap starting.
I20260810 19:35:25.470345 14655 tablet_bootstrap.cc:654] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:25.471593 14656 tablet_bootstrap.cc:654] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:25.471599 14655 log.cc:824] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:25.472139 14654 tablet_bootstrap.cc:492] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: No bootstrap required, opened a new log
I20260810 19:35:25.472455 14654 ts_tablet_manager.cc:1397] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 19:35:25.472815 14656 log.cc:824] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:25.474789 14655 tablet_bootstrap.cc:492] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: No bootstrap required, opened a new log
I20260810 19:35:25.475045 14655 ts_tablet_manager.cc:1397] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260810 19:35:25.475991 14656 tablet_bootstrap.cc:492] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: No bootstrap required, opened a new log
I20260810 19:35:25.476234 14656 ts_tablet_manager.cc:1397] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 19:35:25.476733 14328 heartbeater.cc:503] Master 127.29.233.190:33775 was elected leader, sending a full tablet report...
I20260810 19:35:25.483743 14656 raft_consensus.cc:348] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.483659 14654 raft_consensus.cc:348] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.484082 14656 raft_consensus.cc:374] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:25.484094 14654 raft_consensus.cc:374] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:25.484174 14656 raft_consensus.cc:729] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641dd2cc5c1241c193ff52ea25111e38, State: Initialized, Role: FOLLOWER
I20260810 19:35:25.484184 14654 raft_consensus.cc:729] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c179652fe36f4749bab6ffb0be16e8e8, State: Initialized, Role: FOLLOWER
I20260810 19:35:25.484786 14656 consensus_queue.cc:260] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.484797 14654 consensus_queue.cc:260] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.485878 14633 heartbeater.cc:503] Master 127.29.233.190:33775 was elected leader, sending a full tablet report...
I20260810 19:35:25.486670 14656 ts_tablet_manager.cc:1428] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Time spent starting tablet: real 0.010s user 0.006s sys 0.006s
I20260810 19:35:25.488715 14654 ts_tablet_manager.cc:1428] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260810 19:35:25.489022 14655 raft_consensus.cc:348] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.489427 14655 raft_consensus.cc:374] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:25.489547 14655 raft_consensus.cc:729] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e96e737fa7a4aff9655e603f737fd38, State: Initialized, Role: FOLLOWER
I20260810 19:35:25.490399 14655 consensus_queue.cc:260] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.492352 14480 heartbeater.cc:503] Master 127.29.233.190:33775 was elected leader, sending a full tablet report...
I20260810 19:35:25.492882 14655 ts_tablet_manager.cc:1428] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Time spent starting tablet: real 0.018s user 0.010s sys 0.005s
W20260810 19:35:25.607443 14635 tablet.cc:2369] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:35:25.652697 14481 tablet.cc:2369] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:35:25.691028 14329 tablet.cc:2369] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:35:25.821916 14667 raft_consensus.cc:482] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:35:25.822327 14667 raft_consensus.cc:504] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.824105 14667 leader_election.cc:288] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c179652fe36f4749bab6ffb0be16e8e8 (127.29.233.129:32869), 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253)
I20260810 19:35:25.829909 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.824728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:46397 (local address 127.29.233.130:33253)
0810 19:35:25.824945 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.824951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.824972 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:35:25.825047 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.825050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.825111 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.825227 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:25.825619 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.826523 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.828238 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.829022 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.829181 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.829258 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.829517 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.829642 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:35:25.829923 14638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.824371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33253 (local address 127.29.233.131:46397)
0810 19:35:25.824673 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:25.824695 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:25.824843 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:25.825248 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:25.825252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:25.825279 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:25.825475 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.825484 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.826671 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.826676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.828084 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.828096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.829178 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.829184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.829363 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.829386 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:35:25.829491 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 19:35:25.829632 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260810 19:35:25.830354 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.824480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:44261 (local address 127.29.233.129:32869)
0810 19:35:25.825122 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.825131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.825152 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:35:25.825223 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.825229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.825311 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.825418 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:25.825994 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.826894 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.828644 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.829417 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.829600 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.829668 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.829952 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.830075 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490}
I20260810 19:35:25.830415 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.824404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.29.233.131:44261)
0810 19:35:25.824980 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:25.824997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:25.825098 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:25.825452 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:25.825457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:25.825483 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:25.825829 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.825837 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.827042 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.827057 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.828479 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:25.828490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.829618 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.829624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:25.829786 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.829809 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:35:25.829921 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 19:35:25.830044 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":267,"threads_started":1}
I20260810 19:35:25.831341 14431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57595d7eb78c48bd91ec311dccc76c05" candidate_uuid: "641dd2cc5c1241c193ff52ea25111e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e96e737fa7a4aff9655e603f737fd38" is_pre_election: true
I20260810 19:35:25.831344 14276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57595d7eb78c48bd91ec311dccc76c05" candidate_uuid: "641dd2cc5c1241c193ff52ea25111e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c179652fe36f4749bab6ffb0be16e8e8" is_pre_election: true
I20260810 19:35:25.831846 14431 raft_consensus.cc:2454] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641dd2cc5c1241c193ff52ea25111e38 in term 0.
I20260810 19:35:25.831858 14276 raft_consensus.cc:2454] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641dd2cc5c1241c193ff52ea25111e38 in term 0.
I20260810 19:35:25.832491 14501 leader_election.cc:302] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e96e737fa7a4aff9655e603f737fd38, 641dd2cc5c1241c193ff52ea25111e38; no voters:
I20260810 19:35:25.832898 14667 raft_consensus.cc:2781] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:35:25.833051 14667 raft_consensus.cc:482] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:35:25.833160 14667 raft_consensus.cc:3037] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:25.835453 14667 raft_consensus.cc:504] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.836054 14667 leader_election.cc:288] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [CANDIDATE]: Term 1 election: Requested vote from peers c179652fe36f4749bab6ffb0be16e8e8 (127.29.233.129:32869), 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253)
I20260810 19:35:25.836424 14276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57595d7eb78c48bd91ec311dccc76c05" candidate_uuid: "641dd2cc5c1241c193ff52ea25111e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c179652fe36f4749bab6ffb0be16e8e8"
I20260810 19:35:25.836539 14431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57595d7eb78c48bd91ec311dccc76c05" candidate_uuid: "641dd2cc5c1241c193ff52ea25111e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e96e737fa7a4aff9655e603f737fd38"
I20260810 19:35:25.836642 14276 raft_consensus.cc:3037] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:25.836733 14431 raft_consensus.cc:3037] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:25.839941 14276 raft_consensus.cc:2454] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641dd2cc5c1241c193ff52ea25111e38 in term 1.
I20260810 19:35:25.839936 14431 raft_consensus.cc:2454] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641dd2cc5c1241c193ff52ea25111e38 in term 1.
I20260810 19:35:25.840355 14502 leader_election.cc:302] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e96e737fa7a4aff9655e603f737fd38, 641dd2cc5c1241c193ff52ea25111e38, c179652fe36f4749bab6ffb0be16e8e8; no voters:
I20260810 19:35:25.840693 14667 raft_consensus.cc:2781] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:25.841152 14667 raft_consensus.cc:686] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 LEADER]: Becoming Leader. State: Replica: 641dd2cc5c1241c193ff52ea25111e38, State: Running, Role: LEADER
I20260810 19:35:25.841687 14667 consensus_queue.cc:237] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } }
I20260810 19:35:25.846257 14121 catalog_manager.cc:5495] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641dd2cc5c1241c193ff52ea25111e38 (127.29.233.131). New cstate: current_term: 1 leader_uuid: "641dd2cc5c1241c193ff52ea25111e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "641dd2cc5c1241c193ff52ea25111e38" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 45145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 } health_report { overall_health: UNKNOWN } } }
I20260810 19:35:25.893288 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.887805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.29.233.131:45145)
0810 19:35:25.888041 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.888045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.888066 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:35:25.888153 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.888156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.888234 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.888430 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:35:25.888798 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.889578 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.891013 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.891512 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.891612 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.892068 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:35:25.892588 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:35:25.892697 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.892898 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.893005 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:35:25.901626 14276 raft_consensus.cc:1261] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 FOLLOWER]: Refusing update from remote peer 641dd2cc5c1241c193ff52ea25111e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:25.901657 14431 raft_consensus.cc:1261] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Refusing update from remote peer 641dd2cc5c1241c193ff52ea25111e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:25.902631 14672 consensus_queue.cc:1035] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e96e737fa7a4aff9655e603f737fd38" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:25.903000 14667 consensus_queue.cc:1035] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c179652fe36f4749bab6ffb0be16e8e8" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 32869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:25.914592 14675 mvcc.cc:205] Tried to move back new op lower bound from 7317055594083102720 to 7317055593851867136. Current Snapshot: MvccSnapshot[applied={T|T < 7317055594083102720}]
I20260810 19:35:25.921663 14677 mvcc.cc:205] Tried to move back new op lower bound from 7317055594083102720 to 7317055593851867136. Current Snapshot: MvccSnapshot[applied={T|T < 7317055594083102720}]
I20260810 19:35:25.936736 14122 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:25.937062 14122 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:25.941025 14122 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Sending DeleteTablet for 3 replicas of tablet 57595d7eb78c48bd91ec311dccc76c05
I20260810 19:35:25.942229 14555 tablet_service.cc:1514] Processing DeleteTablet for tablet 57595d7eb78c48bd91ec311dccc76c05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:25 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:60912
I20260810 19:35:25.942260 14409 tablet_service.cc:1514] Processing DeleteTablet for tablet 57595d7eb78c48bd91ec311dccc76c05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:25 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:54080
I20260810 19:35:25.942694 14254 tablet_service.cc:1514] Processing DeleteTablet for tablet 57595d7eb78c48bd91ec311dccc76c05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:25 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:49590
I20260810 19:35:25.943789 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 14161
I20260810 19:35:25.953779 14689 tablet_replica.cc:330] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: stopping tablet replica
I20260810 19:35:25.954427 14689 raft_consensus.cc:2229] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:25.955049 14689 raft_consensus.cc:2258] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:25.957149 14688 tablet_replica.cc:330] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: stopping tablet replica
I20260810 19:35:25.957427 14687 tablet_replica.cc:330] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: stopping tablet replica
I20260810 19:35:25.957858 14687 raft_consensus.cc:2229] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:35:25.958206 14689 ts_tablet_manager.cc:1905] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:25.958576 14687 raft_consensus.cc:2258] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:25.960028 14688 raft_consensus.cc:2229] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:25.960547 14688 raft_consensus.cc:2258] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:25.960840 14687 ts_tablet_manager.cc:1905] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:25.964020 14688 ts_tablet_manager.cc:1905] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:25.967990 14687 ts_tablet_manager.cc:1918] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:25.968181 14687 log.cc:1196] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/57595d7eb78c48bd91ec311dccc76c05
I20260810 19:35:25.968838 14687 ts_tablet_manager.cc:1939] T 57595d7eb78c48bd91ec311dccc76c05 P 641dd2cc5c1241c193ff52ea25111e38: Deleting consensus metadata
I20260810 19:35:25.970688 14106 catalog_manager.cc:4841] TS 641dd2cc5c1241c193ff52ea25111e38 (127.29.233.131:45145): tablet 57595d7eb78c48bd91ec311dccc76c05 (table test-table [id=cd5a9e92766d4b05871f8f21cee0eb38]) successfully deleted
I20260810 19:35:25.975462 14689 ts_tablet_manager.cc:1918] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:25.975485 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:25.945912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49602 (local address 127.29.233.129:32869)
0810 19:35:25.946088 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:25.946095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:25.956359 (+ 10264us) server_negotiation.cc:407] Connection header received
0810 19:35:25.956601 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:25.956607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:25.956697 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:25.956802 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:25.959991 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.961127 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.963038 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:25.963662 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:25.963796 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:25.967435 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:25.967450 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:25.967470 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:25.967515 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:25.971808 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:25.972787 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:25.972797 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:25.972801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:25.972992 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:25.973458 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:25.973467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:25.973471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:25.973995 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:25.974355 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:25.974811 (+ 456us) server_negotiation.cc:299] Negotiation successful
0810 19:35:25.975066 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 19:35:25.975658 14689 log.cc:1196] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/57595d7eb78c48bd91ec311dccc76c05
I20260810 19:35:25.976249 14689 ts_tablet_manager.cc:1939] T 57595d7eb78c48bd91ec311dccc76c05 P c179652fe36f4749bab6ffb0be16e8e8: Deleting consensus metadata
I20260810 19:35:25.976281 14322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:25.979432 14688 ts_tablet_manager.cc:1918] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:25.979677 14688 log.cc:1196] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/57595d7eb78c48bd91ec311dccc76c05
I20260810 19:35:25.980520 14688 ts_tablet_manager.cc:1939] T 57595d7eb78c48bd91ec311dccc76c05 P 2e96e737fa7a4aff9655e603f737fd38: Deleting consensus metadata
I20260810 19:35:25.982920 14107 catalog_manager.cc:4841] TS 2e96e737fa7a4aff9655e603f737fd38 (127.29.233.130:33253): tablet 57595d7eb78c48bd91ec311dccc76c05 (table test-table [id=cd5a9e92766d4b05871f8f21cee0eb38]) successfully deleted
I20260810 19:35:26.071194 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 14161
I20260810 19:35:26.099900 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 14332
W20260810 19:35:26.099994 14108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.233.129:32869 (error 108)
I20260810 19:35:26.105188 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.103761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49610)
0810 19:35:26.104807 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:26.104952 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":800,"threads_started":1}
W20260810 19:35:26.105566 14108 catalog_manager.cc:4584] TS c179652fe36f4749bab6ffb0be16e8e8 (127.29.233.129:32869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57595d7eb78c48bd91ec311dccc76c05: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
I20260810 19:35:26.116556 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.100875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.29.233.130:33253)
0810 19:35:26.101091 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:26.101098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:26.101267 (+ 169us) server_negotiation.cc:407] Connection header received
0810 19:35:26.101473 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:26.101479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:26.101573 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:26.101706 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:26.103172 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:26.104343 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:26.106186 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:26.106856 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:26.107007 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:26.109641 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:26.109658 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:26.109677 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:26.109718 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:26.113529 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:26.114259 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:26.114268 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:26.114273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:26.114540 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:26.114933 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:26.114938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:26.114942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:26.115318 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:26.115631 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:26.115949 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 19:35:26.116179 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 19:35:26.117210 14475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:26.159389 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.158852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49614)
0810 19:35:26.159082 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:26.159207 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 19:35:26.160588 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.160177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49618)
0810 19:35:26.160347 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:26.160433 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 19:35:26.214555 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 14332
I20260810 19:35:26.221244 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.220762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49632)
0810 19:35:26.221003 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:26.221092 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 19:35:26.222465 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.222009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49638)
0810 19:35:26.222165 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:26.222251 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 19:35:26.241120 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 14484
I20260810 19:35:26.257736 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.241997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.29.233.131:45145)
0810 19:35:26.242195 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:26.242205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:26.242235 (+ 30us) server_negotiation.cc:407] Connection header received
0810 19:35:26.242440 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:26.242446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:26.242530 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:26.242636 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:26.244033 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:26.245279 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:26.246795 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:26.247498 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:26.247645 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:26.250294 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:26.250328 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:26.250350 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:26.250405 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:26.254752 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:26.255482 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:26.255492 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:26.255495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:26.255711 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:26.256088 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:26.256093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:26.256096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:26.256562 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:26.256891 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:26.257064 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 19:35:26.257285 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 19:35:26.258584 14628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:26.303337 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.302757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49654)
0810 19:35:26.303026 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:26.303140 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 19:35:26.304579 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:26.304095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:32869 (local address 127.0.0.1:49658)
0810 19:35:26.304316 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:26.304409 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:32869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260810 19:35:26.382951 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 14484
I20260810 19:35:26.406740 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 14086
I20260810 19:35:26.407894 14154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:26.523609 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 14086
I20260810 19:35:26.545518 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33775
--webserver_interface=127.29.233.190
--webserver_port=41707
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:33775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:26.891677 14699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:26.891963 14699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:26.892114 14699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:26.899326 14699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:26.899425 14699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:26.899523 14699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:26.899605 14699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:26.899708 14699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:26.909324 14699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:33775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:33775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=41707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:26.910692 14699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:26.912052 14699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:26.923895 14706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:26.924168 14705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:26.924502 14710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:26.925091 14699 server_base.cc:1034] running on GCE node
I20260810 19:35:26.926051 14699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:26.927678 14699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:26.928893 14699 hybrid_clock.cc:648] HybridClock initialized: now 1786390526928825 us; error 46 us; skew 500 ppm
I20260810 19:35:26.929250 14699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:26 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390526, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:26.938522 14699 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:26.940024 14699 webserver.cc:460] Webserver started at http://127.29.233.190:41707/ using document root <none> and password file <none>
I20260810 19:35:26.940701 14699 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:26.940805 14699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:26.946434 14699 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 19:35:26.949189 14717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:26.950666 14699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:26.950809 14699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "0f4defe67901483ab1d0bd22d8977012"
format_stamp: "Formatted at 2026-08-10 19:35:20 on dist-test-slave-tm1g"
server_key: "fee6a739fad2b4bec9d776a26712b9eb"
server_key_iv: "f75245f94912eb63ea4515893074e27f"
server_key_version: "encryptionkey@0"
I20260810 19:35:26.951761 14699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:26.983433 14699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:26.988161 14699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:26.988407 14699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:27.008205 14699 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:33775
I20260810 19:35:27.008213 14773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:33775 every 8 connection(s)
I20260810 19:35:27.009716 14699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:27.014564 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 14699
I20260810 19:35:27.015430 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:32869
--local_ip_for_outbound_sockets=127.29.233.129
--tserver_master_addrs=127.29.233.190:33775
--webserver_port=34285
--webserver_interface=127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:35:27.016773 14774 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:35:27.020082 14774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Bootstrap starting.
I20260810 19:35:27.041177 14774 log.cc:824] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:27.059140 14774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:35:27.059722 14774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012: Bootstrap complete.
I20260810 19:35:27.069000 14774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:27.069861 14774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f4defe67901483ab1d0bd22d8977012, State: Initialized, Role: FOLLOWER
I20260810 19:35:27.070600 14774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:27.070847 14774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:35:27.070969 14774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:35:27.071159 14774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 2 FOLLOWER]: Advancing to term 3
I20260810 19:35:27.075032 14774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:27.075549 14774 leader_election.cc:302] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f4defe67901483ab1d0bd22d8977012; no voters:
I20260810 19:35:27.076218 14774 leader_election.cc:288] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 19:35:27.076354 14778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 19:35:27.077546 14778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [term 3 LEADER]: Becoming Leader. State: Replica: 0f4defe67901483ab1d0bd22d8977012, State: Running, Role: LEADER
I20260810 19:35:27.078155 14778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } }
I20260810 19:35:27.078465 14774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:27.080830 14780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f4defe67901483ab1d0bd22d8977012. Latest consensus state: current_term: 3 leader_uuid: "0f4defe67901483ab1d0bd22d8977012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } } }
I20260810 19:35:27.081347 14780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:27.081470 14779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0f4defe67901483ab1d0bd22d8977012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4defe67901483ab1d0bd22d8977012" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 33775 } } }
I20260810 19:35:27.081840 14779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4defe67901483ab1d0bd22d8977012 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:27.095489 14786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:27.118659 14786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:27.120149 14786 catalog_manager.cc:1259] Loaded cluster ID: b48b99d5f52f438d9cdb6c98cf4fb738
I20260810 19:35:27.120311 14786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 19:35:27.122387 14786 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd8da1e59ad google::LogMessage::Fail() at ??:0
@ 0x7fd8da1e888a google::LogMessage::SendToLog() at ??:0
@ 0x7fd8da1e54de google::LogMessage::Flush() at ??:0
@ 0x7fd8da1e6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd8efc18512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd8efc15976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
I20260810 19:35:27.165688 14797 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fd8efc9d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd8efc9d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd8dce31345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd8dce3491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd8dce34799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd8dce0feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd8e00e96db start_thread at ??:0
@ 0x7fd8d907f61f clone at ??:0
W20260810 19:35:27.449708 14776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:27.449990 14776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:27.450088 14776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:27.457237 14776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:27.457348 14776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:27.457551 14776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:35:27.467175 14776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:32869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=34285
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:27.468484 14776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:27.469725 14776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:27.483043 14819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:27.483073 14820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:27.483127 14822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:27.484319 14776 server_base.cc:1034] running on GCE node
I20260810 19:35:27.484982 14776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:27.485946 14776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:27.487180 14776 hybrid_clock.cc:648] HybridClock initialized: now 1786390527487138 us; error 55 us; skew 500 ppm
I20260810 19:35:27.487497 14776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:27 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390527, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:27.490737 14776 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:35:27.492443 14776 webserver.cc:460] Webserver started at http://127.29.233.129:34285/ using document root <none> and password file <none>
I20260810 19:35:27.493161 14776 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:27.493348 14776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:27.499006 14776 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 19:35:27.501737 14831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:27.503088 14776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:27.503274 14776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "c179652fe36f4749bab6ffb0be16e8e8"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "1fbe0b240f7c5e3a827f87561dd74bd8"
server_key_iv: "435eebca3c55285f459586fac3d0f57c"
server_key_version: "encryptionkey@0"
I20260810 19:35:27.504320 14776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:27.516848 14776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:27.520998 14776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:27.521291 14776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:27.522286 14776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:27.523653 14776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:27.523789 14776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:27.523922 14776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:27.524013 14776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:27.559897 14776 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:32869
I20260810 19:35:27.559988 14952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:32869 every 8 connection(s)
I20260810 19:35:27.561431 14776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:35:27.566273 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:27.564468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:34353)
0810 19:35:27.565195 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:27.565436 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":289,"threads_started":1}
W20260810 19:35:27.567029 14953 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:33775 (0 consecutive failures): Network error: Failed to ping master at 127.29.233.190:33775: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
I20260810 19:35:27.567896 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:27.567520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:48405)
0810 19:35:27.567690 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:27.567783 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 19:35:27.568794 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:27.568467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:54411)
0810 19:35:27.568604 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:27.568683 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 19:35:27.569106 14953 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 19:35:27.569751 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 14776
I20260810 19:35:27.570478 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:33253
--local_ip_for_outbound_sockets=127.29.233.130
--tserver_master_addrs=127.29.233.190:33775
--webserver_port=39683
--webserver_interface=127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:27.921573 14956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:27.921875 14956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:27.922039 14956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:27.929437 14956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:27.929541 14956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:27.929768 14956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:35:27.939867 14956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:33253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=39683
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:27.941253 14956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:27.942641 14956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:27.955924 14962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:27.956080 14965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:27.956085 14963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:27.957078 14956 server_base.cc:1034] running on GCE node
I20260810 19:35:27.957628 14956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:27.958624 14956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:27.959853 14956 hybrid_clock.cc:648] HybridClock initialized: now 1786390527959797 us; error 53 us; skew 500 ppm
I20260810 19:35:27.960175 14956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:27 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390527, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:27.963348 14956 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:35:27.964830 14956 webserver.cc:460] Webserver started at http://127.29.233.130:39683/ using document root <none> and password file <none>
I20260810 19:35:27.965565 14956 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:27.965718 14956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:27.971134 14956 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:35:27.973734 14973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:27.975154 14956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:35:27.975332 14956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "2e96e737fa7a4aff9655e603f737fd38"
format_stamp: "Formatted at 2026-08-10 19:35:21 on dist-test-slave-tm1g"
server_key: "90f521e9af4905747a6b6f77623ef1d1"
server_key_iv: "8f1615f43350a8908dc66eaa2cc57b2f"
server_key_version: "encryptionkey@0"
I20260810 19:35:27.976318 14956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:28.000085 14956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:28.004483 14956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:28.004717 14956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:28.005703 14956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:28.007308 14956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:28.007428 14956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:28.007567 14956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:28.007650 14956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:28.046027 14956 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:33253
I20260810 19:35:28.046123 15088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:33253 every 8 connection(s)
I20260810 19:35:28.047586 14956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:35:28.050526 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 14956
I20260810 19:35:28.051221 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:45145
--local_ip_for_outbound_sockets=127.29.233.131
--tserver_master_addrs=127.29.233.190:33775
--webserver_port=40543
--webserver_interface=127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:35:28.052732 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:28.050721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:32873)
0810 19:35:28.051560 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:28.051855 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":342,"threads_started":1}
W20260810 19:35:28.053380 15089 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:33775 (0 consecutive failures): Network error: Failed to ping master at 127.29.233.190:33775: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
I20260810 19:35:28.054220 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:28.053885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:54767)
0810 19:35:28.054041 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:28.054114 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 19:35:28.055132 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:28.054808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:34629)
0810 19:35:28.054959 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:28.055029 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 19:35:28.055401 15089 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 19:35:28.392328 15092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:28.392663 15092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:28.392838 15092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:28.400370 15092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:28.400519 15092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:28.400704 15092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:35:28.410787 15092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:45145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=40543
--tserver_master_addrs=127.29.233.190:33775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:28.412227 15092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:28.413625 15092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:28.426851 15098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:28.426872 15099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:28.426947 15101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:28.428100 15092 server_base.cc:1034] running on GCE node
I20260810 19:35:28.428707 15092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:28.429625 15092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:28.430826 15092 hybrid_clock.cc:648] HybridClock initialized: now 1786390528430782 us; error 32 us; skew 500 ppm
I20260810 19:35:28.431169 15092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:28 dist-test-slave-tm1g krb5kdc[13478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390528, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:28.434430 15092 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:35:28.435880 15092 webserver.cc:460] Webserver started at http://127.29.233.131:40543/ using document root <none> and password file <none>
I20260810 19:35:28.436825 15092 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:28.437004 15092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:28.442615 15092 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 19:35:28.445271 15110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:28.446697 15092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:35:28.446879 15092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "641dd2cc5c1241c193ff52ea25111e38"
format_stamp: "Formatted at 2026-08-10 19:35:22 on dist-test-slave-tm1g"
server_key: "84644474175adfc3c2398742e480d651"
server_key_iv: "807d4fa0bed29d28854a3e21e412235a"
server_key_version: "encryptionkey@0"
I20260810 19:35:28.447865 15092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:28.465966 15092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:28.470268 15092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:28.470633 15092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:28.471689 15092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:28.473067 15092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:28.473206 15092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:28.473340 15092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:28.473407 15092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:28.508244 15092 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:45145
I20260810 19:35:28.508296 15228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:45145 every 8 connection(s)
I20260810 19:35:28.509780 15092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:35:28.512010 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 15092
I20260810 19:35:28.515122 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:28.512883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:48739)
0810 19:35:28.513743 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:28.514015 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":399,"threads_started":1}
W20260810 19:35:28.515806 15229 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:33775 (0 consecutive failures): Network error: Failed to ping master at 127.29.233.190:33775: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
I20260810 19:35:28.516598 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:28.516270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:54713)
0810 19:35:28.516409 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:28.516494 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 19:35:28.517407 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:28.517073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:47683)
0810 19:35:28.517212 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:28.517285 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 19:35:28.517691 15229 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 19:35:28.570490 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:28.569582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:60377)
0810 19:35:28.570070 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:28.570176 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":243,"threads_started":1}
I20260810 19:35:29.056749 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:29.055917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:60221)
0810 19:35:29.056363 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:29.056512 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":218,"threads_started":1}
I20260810 19:35:29.519104 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:29.518196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:56541)
0810 19:35:29.518735 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:29.518872 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":289,"threads_started":1}
I20260810 19:35:29.572252 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:29.571462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:53811)
0810 19:35:29.571941 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:29.572026 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":236,"threads_started":1}
I20260810 19:35:30.058704 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:30.057785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:34409)
0810 19:35:30.058395 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:30.058510 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":245,"threads_started":1}
I20260810 19:35:30.520773 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:30.520010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:48167)
0810 19:35:30.520503 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:30.520590 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":233,"threads_started":1}
I20260810 19:35:30.573791 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:30.573089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:35515)
0810 19:35:30.573530 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:30.573611 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260810 19:35:31.060326 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:31.059629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:52191)
0810 19:35:31.060066 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:31.060144 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":234,"threads_started":1}
I20260810 19:35:31.522727 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:31.521849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:52529)
0810 19:35:31.522420 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:31.522553 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":330,"threads_started":1}
I20260810 19:35:31.575654 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:31.574880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:33457)
0810 19:35:31.575337 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:31.575420 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":236,"threads_started":1}
I20260810 19:35:32.062089 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:32.061295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:38095)
0810 19:35:32.061804 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:32.061900 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":261,"threads_started":1}
I20260810 19:35:32.524380 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:32.523615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:40565)
0810 19:35:32.524098 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:32.524180 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":285,"threads_started":1}
I20260810 19:35:32.577335 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:32.576601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:56537)
0810 19:35:32.577072 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:32.577159 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":244,"threads_started":1}
I20260810 19:35:33.063850 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:33.063092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:38859)
0810 19:35:33.063597 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:33.063681 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":263,"threads_started":1}
I20260810 19:35:33.526386 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:33.525363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:49093)
0810 19:35:33.525984 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:33.526110 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":303,"threads_started":1}
I20260810 19:35:33.579092 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:33.578271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:35353)
0810 19:35:33.578817 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:33.578924 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":263,"threads_started":1}
I20260810 19:35:34.065634 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:34.064865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:52269)
0810 19:35:34.065365 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:34.065464 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":248,"threads_started":1}
I20260810 19:35:34.527966 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:34.527223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:54951)
0810 19:35:34.527682 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:34.527783 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":236,"threads_started":1}
I20260810 19:35:34.580878 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:34.580052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:33199)
0810 19:35:34.580552 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:34.580639 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":251,"threads_started":1}
I20260810 19:35:35.067709 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:35.066764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:57147)
0810 19:35:35.067352 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:35.067473 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":292,"threads_started":1}
I20260810 19:35:35.529798 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:35.528965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:52467)
0810 19:35:35.529490 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:35.529574 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":236,"threads_started":1}
I20260810 19:35:35.582748 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:35.581926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:37965)
0810 19:35:35.582481 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:35.582577 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":251,"threads_started":1}
I20260810 19:35:36.069468 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:36.068653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:56403)
0810 19:35:36.069204 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:36.069293 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":251,"threads_started":1}
I20260810 19:35:36.531533 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:36.530827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:56287)
0810 19:35:36.531262 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:36.531366 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":221,"threads_started":1}
I20260810 19:35:36.584509 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:36.583731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:55219)
0810 19:35:36.584230 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:36.584317 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":242,"threads_started":1}
I20260810 19:35:37.071190 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:37.070425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:43257)
0810 19:35:37.070920 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:37.071004 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":230,"threads_started":1}
I20260810 19:35:37.533196 15262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:37.532468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:37503)
0810 19:35:37.532938 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:37.533018 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":222,"threads_started":1}
I20260810 19:35:37.586437 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:37.585594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:50477)
0810 19:35:37.586132 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:37.586212 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":322,"threads_started":1}
I20260810 19:35:38.072981 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:38.072195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:50507)
0810 19:35:38.072708 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:38.072807 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":292,"threads_started":1}
I20260810 19:35:38.535251 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:38.534429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:47551)
0810 19:35:38.534994 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:38.535083 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":312,"threads_started":1}
I20260810 19:35:38.588119 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:38.587329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:41727)
0810 19:35:38.587826 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:38.587934 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":255,"threads_started":1}
I20260810 19:35:39.074726 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:39.073934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:40085)
0810 19:35:39.074438 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:39.074542 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":227,"threads_started":1}
I20260810 19:35:39.536952 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:39.536179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:52393)
0810 19:35:39.536663 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:39.536760 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":237,"threads_started":1}
I20260810 19:35:39.589787 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:39.589074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:35845)
0810 19:35:39.589540 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:39.589627 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":230,"threads_started":1}
I20260810 19:35:40.076449 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:40.075683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:41663)
0810 19:35:40.076189 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:40.076274 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":253,"threads_started":1}
I20260810 19:35:40.538751 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:40.537923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:52549)
0810 19:35:40.538486 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:40.538574 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":263,"threads_started":1}
I20260810 19:35:40.591605 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:40.590815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:53297)
0810 19:35:40.591299 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:40.591415 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":243,"threads_started":1}
I20260810 19:35:41.078418 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:41.077458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:47507)
0810 19:35:41.078024 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:41.078172 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":296,"threads_started":1}
I20260810 19:35:41.540573 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:41.539770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:37159)
0810 19:35:41.540288 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:41.540383 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":276,"threads_started":1}
I20260810 19:35:41.593302 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:41.592549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:48501)
0810 19:35:41.593017 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:41.593115 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
I20260810 19:35:42.080410 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:42.079549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:45755)
0810 19:35:42.080099 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:42.080216 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":269,"threads_started":1}
I20260810 19:35:42.542599 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:42.541643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:34143)
0810 19:35:42.542167 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:42.542355 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":281,"threads_started":1}
I20260810 19:35:42.595489 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:42.594693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:35901)
0810 19:35:42.595159 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:42.595267 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":237,"threads_started":1}
I20260810 19:35:43.082585 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:43.081613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.130:42343)
0810 19:35:43.082190 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:43.082371 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
I20260810 19:35:43.518775 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 14776
I20260810 19:35:43.544745 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:43.543774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.131:58929)
0810 19:35:43.544261 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:43.544478 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":242,"threads_started":1}
I20260810 19:35:43.554005 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:43.519965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.29.233.129:32869)
0810 19:35:43.520718 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:43.520733 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:43.520773 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:35:43.523198 (+ 2425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:43.523208 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:43.523646 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:43.524018 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:43.525603 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:43.526969 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:43.528341 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:43.529021 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:43.529224 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:43.547066 (+ 17842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:43.547077 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:43.547097 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:43.547149 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:43.550225 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:43.551106 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:43.551116 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:43.551126 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:43.551325 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:43.551751 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:43.551756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:43.551761 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:43.552411 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:43.552739 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:43.553249 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 19:35:43.553707 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":338,"threads_started":1}
I20260810 19:35:43.555594 14947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:43.643479 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:43.642417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:33775 (local address 127.29.233.129:59475)
0810 19:35:43.642973 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:43.643199 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":270,"threads_started":1}
I20260810 19:35:43.644398 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 14776
I20260810 19:35:43.664522 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 14956
I20260810 19:35:43.683838 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:43.665565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.29.233.130:33253)
0810 19:35:43.666142 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:43.666157 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:43.666197 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:35:43.667972 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:43.667983 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:43.668455 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:43.668840 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:43.670562 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:43.671867 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:43.673223 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:43.673910 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:43.674072 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:43.676934 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:43.676941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:43.676956 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:43.676998 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:43.680413 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:43.681196 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:43.681203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:43.681209 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:43.681386 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:43.681713 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:43.681717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:43.681719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:43.682278 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:43.682604 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:43.682950 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 19:35:43.683533 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":284,"threads_started":1}
I20260810 19:35:43.685393 15084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:43.775107 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 14956
I20260810 19:35:43.797605 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 15092
I20260810 19:35:43.816498 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:43.798702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.29.233.131:45145)
0810 19:35:43.799283 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:43.799299 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:43.799350 (+ 51us) server_negotiation.cc:407] Connection header received
0810 19:35:43.801047 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:43.801058 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:43.801517 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:43.801822 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:43.803527 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:43.804762 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:43.806029 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:43.806805 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:43.806950 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:43.809677 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:43.809686 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:43.809701 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:43.809746 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:43.813075 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:43.813853 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:43.813860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:43.813866 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:43.814054 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:43.814537 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:43.814541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:43.814543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:43.815102 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:43.815470 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:43.815833 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 19:35:43.816192 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":259,"threads_started":1}
I20260810 19:35:43.817984 15224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:43.902661 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 15092
2026-08-10T19:35:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25072 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:35:44 dist-test-slave-tm1g krb5kdc[15294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:35:44 dist-test-slave-tm1g krb5kdc[15294](info): set up 2 sockets
Aug 10 19:35:44 dist-test-slave-tm1g krb5kdc[15294](info): commencing operation
krb5kdc: starting...
W20260810 19:35:45.101166 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:35:45 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:35:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:35:45.438643 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42273
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:42273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:45.807677 15307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:45.807982 15307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:45.808187 15307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:45.815518 15307 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 19:35:45.815618 15307 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 19:35:45.815687 15307 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 19:35:45.815765 15307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:35:45.815860 15307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:45.815997 15307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:45.816072 15307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:35:45.816161 15307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:35:45.826215 15307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:42273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:45.827654 15307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:45.829085 15307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:45.842684 15312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:45.842604 15313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:45.842645 15315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:45.843534 15307 server_base.cc:1034] running on GCE node
I20260810 19:35:45.844401 15307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:45.845983 15307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:45.847265 15307 hybrid_clock.cc:648] HybridClock initialized: now 1786390545847204 us; error 51 us; skew 500 ppm
I20260810 19:35:45.847661 15307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:45 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390545, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:45.851258 15307 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:35:45.852808 15307 webserver.cc:460] Webserver started at http://127.29.233.190:33557/ using document root <none> and password file <none>
I20260810 19:35:45.853582 15307 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:45.853680 15307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:45.853987 15307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:45.857368 15307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "4d568c2817be415290e81e59c020a5e9"
format_stamp: "Formatted at 2026-08-10 19:35:45 on dist-test-slave-tm1g"
server_key: "88aad8a079effe6a096515101e9c192f"
server_key_iv: "ff4551a5b7d729d04f1e2e6dd5c989d5"
server_key_version: "encryptionkey@0"
I20260810 19:35:45.858215 15307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "4d568c2817be415290e81e59c020a5e9"
format_stamp: "Formatted at 2026-08-10 19:35:45 on dist-test-slave-tm1g"
server_key: "88aad8a079effe6a096515101e9c192f"
server_key_iv: "ff4551a5b7d729d04f1e2e6dd5c989d5"
server_key_version: "encryptionkey@0"
I20260810 19:35:45.863988 15307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:35:45.867734 15322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:45.869289 15307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:35:45.869462 15307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "4d568c2817be415290e81e59c020a5e9"
format_stamp: "Formatted at 2026-08-10 19:35:45 on dist-test-slave-tm1g"
server_key: "88aad8a079effe6a096515101e9c192f"
server_key_iv: "ff4551a5b7d729d04f1e2e6dd5c989d5"
server_key_version: "encryptionkey@0"
I20260810 19:35:45.869665 15307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:45.894068 15307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:45.898828 15307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:45.899089 15307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:45.919066 15307 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:42273
I20260810 19:35:45.919087 15376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:42273 every 8 connection(s)
I20260810 19:35:45.920661 15307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:35:45.922583 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 15307
I20260810 19:35:45.922855 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:35:45.923764 30630 external_mini_cluster.cc:1428] Setting key a280f28a53c5d440234f3f3a34b63305
I20260810 19:35:45.925858 15377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 19:35:45 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:45.941618 15377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Bootstrap starting.
I20260810 19:35:45.945530 15377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:45.946733 15377 log.cc:824] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:45.949756 15380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:45.926406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.29.233.190:42273)
0810 19:35:45.927336 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:45.927349 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:45.927395 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:35:45.929196 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:45.929220 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:45.929665 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:45.930069 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:45.932151 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:45.933649 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:45.935628 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:45.936356 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:45.936500 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:45.941914 (+ 5414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:45.941928 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:45.941969 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:45.942041 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:45.945233 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:45.946116 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:45.946123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:45.946129 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:45.946296 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:45.946770 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:45.946775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:45.946777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:45.947419 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:45.947792 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:45.948287 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 19:35:45.948741 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":454,"threads_started":1}
I20260810 19:35:45.950068 15377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: No bootstrap required, opened a new log
I20260810 19:35:45.958077 15377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:35:45.958698 15377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:45.958853 15377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d568c2817be415290e81e59c020a5e9, State: Initialized, Role: FOLLOWER
I20260810 19:35:45.959529 15377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:35:45.959812 15377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:35:45.959968 15377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:35:45.960183 15377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:45.962596 15377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:35:45.963194 15377 leader_election.cc:302] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d568c2817be415290e81e59c020a5e9; no voters:
I20260810 19:35:45.963999 15377 leader_election.cc:288] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:35:45.964171 15382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:45.964921 15382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 LEADER]: Becoming Leader. State: Replica: 4d568c2817be415290e81e59c020a5e9, State: Running, Role: LEADER
I20260810 19:35:45.965595 15382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:35:45.966290 15377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:35:45.968947 15384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d568c2817be415290e81e59c020a5e9. Latest consensus state: current_term: 1 leader_uuid: "4d568c2817be415290e81e59c020a5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } } }
I20260810 19:35:45.969030 15383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d568c2817be415290e81e59c020a5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } } }
I20260810 19:35:45.969409 15384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:45.969465 15383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: This master's current role is: LEADER
I20260810 19:35:45.971246 15390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:35:45.977226 15390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:35:45.986253 15390 catalog_manager.cc:1347] Generated new cluster ID: 0ef9c54e8f694005860496fe78406f85
I20260810 19:35:45.986414 15390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:35:45.997740 15390 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:35:45.999205 15390 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:35:46.010294 15390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Generated new TSK 0
I20260810 19:35:46.011206 15390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:35:46.092427 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42273
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:46.451274 15406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:46.451632 15406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:46.451817 15406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:46.459359 15406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:46.459527 15406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:46.459721 15406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:35:46.470162 15406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:46.471692 15406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:46.473250 15406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:46.487550 15411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:46.487483 15415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:46.487509 15412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:46.489212 15406 server_base.cc:1034] running on GCE node
I20260810 19:35:46.489881 15406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:46.490998 15406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:46.492240 15406 hybrid_clock.cc:648] HybridClock initialized: now 1786390546492189 us; error 36 us; skew 500 ppm
I20260810 19:35:46.492619 15406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:46 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390546, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:46.496219 15406 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:35:46.498044 15406 webserver.cc:460] Webserver started at http://127.29.233.129:41905/ using document root <none> and password file <none>
I20260810 19:35:46.498878 15406 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:46.499022 15406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:46.499359 15406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:46.502514 15406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "b7981fec0ee443c3a42ffa76d3f60ca1"
format_stamp: "Formatted at 2026-08-10 19:35:46 on dist-test-slave-tm1g"
server_key: "b06bc8c2ab1628cd2005bb15b231aa31"
server_key_iv: "799477e748c9dc39a09fecad1f0f309b"
server_key_version: "encryptionkey@0"
I20260810 19:35:46.503298 15406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "b7981fec0ee443c3a42ffa76d3f60ca1"
format_stamp: "Formatted at 2026-08-10 19:35:46 on dist-test-slave-tm1g"
server_key: "b06bc8c2ab1628cd2005bb15b231aa31"
server_key_iv: "799477e748c9dc39a09fecad1f0f309b"
server_key_version: "encryptionkey@0"
I20260810 19:35:46.508739 15406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 19:35:46.512495 15424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:46.513868 15406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:35:46.514048 15406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "b7981fec0ee443c3a42ffa76d3f60ca1"
format_stamp: "Formatted at 2026-08-10 19:35:46 on dist-test-slave-tm1g"
server_key: "b06bc8c2ab1628cd2005bb15b231aa31"
server_key_iv: "799477e748c9dc39a09fecad1f0f309b"
server_key_version: "encryptionkey@0"
I20260810 19:35:46.514223 15406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:46.548725 15406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:46.553315 15406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:46.553647 15406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:46.554718 15406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:46.556185 15406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:46.556262 15406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:46.556437 15406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:46.556512 15406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:46.591884 15406 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39027
I20260810 19:35:46.591971 15544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39027 every 8 connection(s)
I20260810 19:35:46.593222 15406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:35:46.596889 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 15406
I20260810 19:35:46.597147 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:35:46.598096 30630 external_mini_cluster.cc:1428] Setting key 9a41e2e8813c02e70a2f913f981b801b
Aug 10 19:35:46 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390546, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:46.617105 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:46.595993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:34881 (local address 127.29.233.190:42273)
0810 19:35:46.596563 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:46.596572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:46.597612 (+ 1040us) server_negotiation.cc:407] Connection header received
0810 19:35:46.600064 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:46.600070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:46.600152 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:46.600289 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:46.602557 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:46.603334 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:46.605997 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:46.606488 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:46.606594 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:46.611095 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:46.611101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:46.611104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:46.611141 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:46.613830 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:46.614660 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:46.614670 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:46.614672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:46.614796 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:46.615260 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:46.615265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:46.615268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:46.615558 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:46.615724 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:46.616467 (+ 743us) server_negotiation.cc:299] Negotiation successful
0810 19:35:46.616702 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":268,"threads_started":1}
I20260810 19:35:46.618410 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:46.596530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:34881)
0810 19:35:46.597407 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:46.597469 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:46.599731 (+ 2262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:46.600616 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:46.600629 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:46.601469 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:46.602360 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:46.602390 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:46.603577 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:46.603585 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:46.605829 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:46.605850 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:46.606736 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:46.606746 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:46.607119 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:46.607922 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:46.607976 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:46.610831 (+ 2855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:46.613984 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:46.613994 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:46.614011 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:46.614523 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:46.614911 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:46.614916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:46.614919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:46.615127 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:46.615698 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:46.615706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:46.616065 (+ 359us) client_negotiation.cc:769] Sending connection context
0810 19:35:46.616349 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 19:35:46.616902 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":357,"threads_started":1}
I20260810 19:35:46.620046 15545 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42273
I20260810 19:35:46.620440 15545 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:46.621330 15545 heartbeater.cc:511] Master 127.29.233.190:42273 requested a full tablet report, sending...
I20260810 19:35:46.623677 15339 ts_manager.cc:194] Registered new tserver with Master: b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129:39027)
I20260810 19:35:46.625131 15339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:34881
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:35:46.673271 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42273
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:47.031430 15551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:47.031777 15551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:47.031948 15551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:47.039410 15551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:47.039577 15551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:47.039786 15551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:35:47.049973 15551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:47.051415 15551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:47.052824 15551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:35:47.066725 15557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:47.066895 15556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:47.066847 15560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:47.067744 15551 server_base.cc:1034] running on GCE node
I20260810 19:35:47.068385 15551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:47.069379 15551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:47.070590 15551 hybrid_clock.cc:648] HybridClock initialized: now 1786390547070550 us; error 32 us; skew 500 ppm
I20260810 19:35:47.070926 15551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:47 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390547, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:47.074541 15551 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:35:47.076493 15551 webserver.cc:460] Webserver started at http://127.29.233.130:41693/ using document root <none> and password file <none>
I20260810 19:35:47.077585 15551 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:47.077742 15551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:47.078131 15551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:47.081290 15551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "170056b505eb4a0995cec77493d3fbd2"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "c32887a168936306bc5b399c500d5ced"
server_key_iv: "595c612198651f286ffd8f8f2e4a5926"
server_key_version: "encryptionkey@0"
I20260810 19:35:47.082136 15551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "170056b505eb4a0995cec77493d3fbd2"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "c32887a168936306bc5b399c500d5ced"
server_key_iv: "595c612198651f286ffd8f8f2e4a5926"
server_key_version: "encryptionkey@0"
I20260810 19:35:47.087646 15551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 19:35:47.091326 15568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:47.092708 15551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:35:47.092901 15551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "170056b505eb4a0995cec77493d3fbd2"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "c32887a168936306bc5b399c500d5ced"
server_key_iv: "595c612198651f286ffd8f8f2e4a5926"
server_key_version: "encryptionkey@0"
I20260810 19:35:47.093123 15551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:47.105515 15551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:47.109799 15551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:47.110116 15551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:47.111142 15551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:47.112555 15551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:47.112663 15551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:47.112823 15551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:47.112903 15551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:47.150080 15551 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:40643
I20260810 19:35:47.150171 15689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:40643 every 8 connection(s)
I20260810 19:35:47.151456 15551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:35:47.157189 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 15551
I20260810 19:35:47.157431 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:35:47.157922 30630 external_mini_cluster.cc:1428] Setting key e902ad8b42b9492c967113b67a2776c7
Aug 10 19:35:47 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390547, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:47.175194 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.154169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:41825 (local address 127.29.233.190:42273)
0810 19:35:47.154704 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:47.154709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:47.155692 (+ 983us) server_negotiation.cc:407] Connection header received
0810 19:35:47.157711 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:47.157717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:47.157816 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:47.157959 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:47.160065 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.160900 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.162621 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.163060 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.163195 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.167427 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:47.167433 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:47.167436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:47.167480 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:47.170867 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:47.171951 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:47.171963 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:47.171966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:47.172136 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:47.172713 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:47.172721 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:47.172732 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:47.173164 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:47.173328 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:47.174354 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0810 19:35:47.174676 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":298,"threads_started":1}
I20260810 19:35:47.176067 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.154739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:41825)
0810 19:35:47.155493 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:47.155549 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:47.157376 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:47.158205 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:47.158214 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:47.158983 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:47.159819 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.159858 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.161057 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.161067 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.162457 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.162475 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.163303 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.163310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.163551 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.164261 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:47.164283 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:47.167148 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:47.171047 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:47.171060 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:47.171082 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:47.171770 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:47.172275 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:47.172282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:47.172288 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:47.172549 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:47.173434 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:47.173443 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:47.173890 (+ 447us) client_negotiation.cc:769] Sending connection context
0810 19:35:47.174154 (+ 264us) client_negotiation.cc:240] Negotiation successful
0810 19:35:47.174768 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":300,"threads_started":1}
I20260810 19:35:47.177771 15690 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42273
I20260810 19:35:47.178061 15690 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:47.178831 15690 heartbeater.cc:511] Master 127.29.233.190:42273 requested a full tablet report, sending...
I20260810 19:35:47.180418 15339 ts_manager.cc:194] Registered new tserver with Master: 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643)
I20260810 19:35:47.181309 15339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:41825
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:35:47.231227 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42273
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:35:47.602111 15696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:35:47.602485 15696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:35:47.602658 15696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:35:47.609901 15696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:35:47.610052 15696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:35:47.610245 15696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:35:47.619977 15696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:35:47.621313 15696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:35:47.622680 15696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 19:35:47.629020 15545 heartbeater.cc:503] Master 127.29.233.190:42273 was elected leader, sending a full tablet report...
W20260810 19:35:47.636333 15705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:47.636392 15703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:35:47.636421 15702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:35:47.637322 15696 server_base.cc:1034] running on GCE node
I20260810 19:35:47.637933 15696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:35:47.638971 15696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:35:47.640246 15696 hybrid_clock.cc:648] HybridClock initialized: now 1786390547640204 us; error 63 us; skew 500 ppm
I20260810 19:35:47.640618 15696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:35:47 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390547, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:35:47.643962 15696 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:35:47.645493 15696 webserver.cc:460] Webserver started at http://127.29.233.131:33031/ using document root <none> and password file <none>
I20260810 19:35:47.646380 15696 fs_manager.cc:359] Metadata directory not provided
I20260810 19:35:47.646543 15696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:35:47.646883 15696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:35:47.649962 15696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "d3efd9235f833bfe9a3e839d12995270"
server_key_iv: "4eb3eea796173991080ebb518791da4b"
server_key_version: "encryptionkey@0"
I20260810 19:35:47.650774 15696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "d3efd9235f833bfe9a3e839d12995270"
server_key_iv: "4eb3eea796173991080ebb518791da4b"
server_key_version: "encryptionkey@0"
I20260810 19:35:47.656497 15696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:35:47.660118 15713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:47.661665 15696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:35:47.661847 15696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "d3efd9235f833bfe9a3e839d12995270"
server_key_iv: "4eb3eea796173991080ebb518791da4b"
server_key_version: "encryptionkey@0"
I20260810 19:35:47.662056 15696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:35:47.699579 15696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:35:47.703967 15696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:35:47.704283 15696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:35:47.705304 15696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:35:47.706758 15696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:35:47.706867 15696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:47.707024 15696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:35:47.707101 15696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:35:47.742887 15696 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:37729
I20260810 19:35:47.742985 15834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:37729 every 8 connection(s)
I20260810 19:35:47.744608 15696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:35:47.753885 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 15696
I20260810 19:35:47.754145 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:35:47.754797 30630 external_mini_cluster.cc:1428] Setting key f9c5f30975a911d4b014a9b738b3785a
Aug 10 19:35:47 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390547, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:35:47.771536 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.750640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:57977 (local address 127.29.233.190:42273)
0810 19:35:47.751251 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:47.751261 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:47.752614 (+ 1353us) server_negotiation.cc:407] Connection header received
0810 19:35:47.755463 (+ 2849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:47.755469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:47.755600 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:47.755721 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:47.757800 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.758903 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.760501 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.760945 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.761056 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.764800 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:47.764805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:47.764809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:47.764844 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:47.767779 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:47.768499 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:47.768502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:47.768504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:47.768648 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:47.769102 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:47.769109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:47.769116 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:47.769446 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:47.769619 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:47.770665 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0810 19:35:47.771080 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":276,"threads_started":1}
I20260810 19:35:47.772573 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.749157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:57977)
0810 19:35:47.751739 (+ 2582us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:47.751788 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:47.754961 (+ 3173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:47.756205 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:47.756218 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:47.756854 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:35:47.757609 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.757636 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.759063 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.759067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.760341 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.760353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.761187 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.761192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.761418 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.762149 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:35:47.762178 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:35:47.764539 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:35:47.767914 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:47.767921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:47.767933 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:47.768366 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:47.768761 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:35:47.768765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:35:47.768767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:35:47.768972 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:35:47.769605 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:35:47.769615 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:35:47.770030 (+ 415us) client_negotiation.cc:769] Sending connection context
0810 19:35:47.770448 (+ 418us) client_negotiation.cc:240] Negotiation successful
0810 19:35:47.771133 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2282,"thread_start_us":2038,"threads_started":1}
I20260810 19:35:47.773940 15836 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42273
I20260810 19:35:47.774279 15836 heartbeater.cc:464] Registering TS with master...
I20260810 19:35:47.775161 15836 heartbeater.cc:511] Master 127.29.233.190:42273 requested a full tablet report, sending...
I20260810 19:35:47.776571 15339 ts_manager.cc:194] Registered new tserver with Master: 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131:37729)
I20260810 19:35:47.777374 15339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:57977
I20260810 19:35:47.783946 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:35:47.802489 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.788817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.29.233.190:42273)
0810 19:35:47.789026 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:47.789032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:47.789549 (+ 517us) server_negotiation.cc:407] Connection header received
0810 19:35:47.789680 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:47.789684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:47.789750 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:47.789843 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:47.791446 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.792237 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.793626 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.794021 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.794118 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.796704 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:47.796709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:47.796712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:47.796803 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:47.799804 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:47.800513 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:47.800517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:47.800519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:47.800643 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:47.800975 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:47.800978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:47.800980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:47.801296 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:47.801515 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:47.801955 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 19:35:47.802129 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 19:35:47.806931 15339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:35:47.813987 15339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:35:47.839085 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.827834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:40643 (local address 127.0.0.1:36148)
0810 19:35:47.829057 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:47.829072 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:47.829259 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:47.830345 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:47.830352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:47.830412 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:47.831030 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.831044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.832580 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.832589 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.836010 (+ 3421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.836027 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.837690 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.837696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.837909 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.837957 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 19:35:47.838095 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 19:35:47.838288 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1124,"spinlock_wait_cycles":19328,"thread_start_us":1725,"threads_started":1}
I20260810 19:35:47.839066 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.828181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.29.233.130:40643)
0810 19:35:47.828801 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:47.828811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:47.829466 (+ 655us) server_negotiation.cc:407] Connection header received
0810 19:35:47.829582 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:47.829591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:47.829834 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:47.830022 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:47.831222 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.832440 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.836193 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.837486 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.837678 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.837864 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:47.838138 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 19:35:47.838346 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":310,"threads_started":1}
I20260810 19:35:47.840448 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.827468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.29.233.129:39027)
0810 19:35:47.828452 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:47.828465 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:47.829182 (+ 717us) server_negotiation.cc:407] Connection header received
0810 19:35:47.829526 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:47.829538 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:47.829833 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:47.830023 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:47.833407 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.834928 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.837008 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.838493 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.838713 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.838881 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:47.839895 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0810 19:35:47.840068 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":571,"threads_started":1}
I20260810 19:35:47.840481 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.827375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39027 (local address 127.0.0.1:47730)
0810 19:35:47.829018 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:47.829067 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:47.829260 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:47.830342 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:47.830350 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:47.830395 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:47.833178 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.833194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.835043 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.835048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.836836 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.836851 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.839493 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.839501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.839681 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.839704 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:35:47.839828 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 19:35:47.839959 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1381,"mutex_wait_us":1,"spinlock_wait_cycles":8192,"thread_start_us":1231,"threads_started":1}
I20260810 19:35:47.843039 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.829438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:37729 (local address 127.0.0.1:39022)
0810 19:35:47.833428 (+ 3990us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:47.833446 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:47.833583 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:47.834368 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:47.834373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:47.834398 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:47.834845 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.834860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.836632 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.836638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.840101 (+ 3463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:47.840121 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.842177 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.842184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:47.842382 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.842429 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 19:35:47.842545 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 19:35:47.842683 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3854,"thread_start_us":3694,"threads_started":1}
I20260810 19:35:47.843134 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:47.829880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.29.233.131:37729)
0810 19:35:47.833061 (+ 3181us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:47.833070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:47.833566 (+ 496us) server_negotiation.cc:407] Connection header received
0810 19:35:47.833749 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:47.833758 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:47.834009 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:47.834196 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:47.835036 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.836495 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.840296 (+ 3801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:47.841960 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:47.842177 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:47.842387 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:47.842605 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 19:35:47.842763 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2991,"thread_start_us":2779,"threads_started":1}
I20260810 19:35:47.843735 15619 tablet_service.cc:1467] Processing CreateTablet for tablet 17d1cd034bc747909c7e9619cf03b7f4 (DEFAULT_TABLE table=test-table [id=320c54851d774e899eb728858d989000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:47.844540 15475 tablet_service.cc:1467] Processing CreateTablet for tablet 17d1cd034bc747909c7e9619cf03b7f4 (DEFAULT_TABLE table=test-table [id=320c54851d774e899eb728858d989000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:47.845377 15619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d1cd034bc747909c7e9619cf03b7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:47.846012 15475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d1cd034bc747909c7e9619cf03b7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:47.849334 15767 tablet_service.cc:1467] Processing CreateTablet for tablet 17d1cd034bc747909c7e9619cf03b7f4 (DEFAULT_TABLE table=test-table [id=320c54851d774e899eb728858d989000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:35:47.850764 15767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d1cd034bc747909c7e9619cf03b7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:35:47.860867 15856 tablet_bootstrap.cc:492] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Bootstrap starting.
I20260810 19:35:47.862401 15857 tablet_bootstrap.cc:492] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Bootstrap starting.
I20260810 19:35:47.864401 15858 tablet_bootstrap.cc:492] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Bootstrap starting.
I20260810 19:35:47.866057 15857 tablet_bootstrap.cc:654] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:47.866104 15856 tablet_bootstrap.cc:654] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:47.867458 15857 log.cc:824] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:47.867897 15858 tablet_bootstrap.cc:654] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Neither blocks nor log segments found. Creating new log.
I20260810 19:35:47.868232 15856 log.cc:824] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:47.869314 15858 log.cc:824] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Log is configured to *not* fsync() on all Append() calls
I20260810 19:35:47.871153 15856 tablet_bootstrap.cc:492] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: No bootstrap required, opened a new log
I20260810 19:35:47.871428 15856 ts_tablet_manager.cc:1397] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 19:35:47.871990 15858 tablet_bootstrap.cc:492] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: No bootstrap required, opened a new log
I20260810 19:35:47.872234 15858 ts_tablet_manager.cc:1397] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 19:35:47.879273 15857 tablet_bootstrap.cc:492] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: No bootstrap required, opened a new log
I20260810 19:35:47.879571 15857 ts_tablet_manager.cc:1397] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.001s
I20260810 19:35:47.879920 15858 raft_consensus.cc:348] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:47.880298 15858 raft_consensus.cc:374] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:47.880389 15858 raft_consensus.cc:729] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3ba879ea9d4d89aaf7d0b0f626d0f9, State: Initialized, Role: FOLLOWER
I20260810 19:35:47.880283 15856 raft_consensus.cc:348] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:47.880702 15856 raft_consensus.cc:374] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:47.880795 15856 raft_consensus.cc:729] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170056b505eb4a0995cec77493d3fbd2, State: Initialized, Role: FOLLOWER
I20260810 19:35:47.881040 15858 consensus_queue.cc:260] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:47.881392 15856 consensus_queue.cc:260] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:47.882356 15836 heartbeater.cc:503] Master 127.29.233.190:42273 was elected leader, sending a full tablet report...
I20260810 19:35:47.883069 15690 heartbeater.cc:503] Master 127.29.233.190:42273 was elected leader, sending a full tablet report...
I20260810 19:35:47.883769 15856 ts_tablet_manager.cc:1428] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260810 19:35:47.886905 15858 ts_tablet_manager.cc:1428] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Time spent starting tablet: real 0.015s user 0.011s sys 0.001s
I20260810 19:35:47.890199 15857 raft_consensus.cc:348] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:47.890722 15857 raft_consensus.cc:374] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:35:47.890851 15857 raft_consensus.cc:729] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7981fec0ee443c3a42ffa76d3f60ca1, State: Initialized, Role: FOLLOWER
I20260810 19:35:47.891448 15857 consensus_queue.cc:260] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:47.892843 15857 ts_tablet_manager.cc:1428] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
W20260810 19:35:47.903421 15691 tablet.cc:2369] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:35:48.005007 15839 tablet.cc:2369] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:35:48.081034 15865 raft_consensus.cc:482] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:35:48.081437 15865 raft_consensus.cc:504] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:48.083582 15865 leader_election.cc:288] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129:39027), 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643)
I20260810 19:35:48.090727 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.083917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:48473 (local address 127.29.233.129:39027)
0810 19:35:48.084136 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.084144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.084245 (+ 101us) server_negotiation.cc:407] Connection header received
0810 19:35:48.084488 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.084492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.084554 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.084671 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:48.085230 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.086495 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.088683 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.089874 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.090001 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.090055 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.090436 (+ 381us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.090540 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:35:48.090839 15837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.083877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39027 (local address 127.29.233.131:48473)
0810 19:35:48.084138 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:48.084161 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:48.084346 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:48.084694 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:48.084698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:48.084728 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:48.085007 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.085020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.086638 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.086646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.088501 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.088514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.090034 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.090041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.090238 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.090272 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 19:35:48.090408 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 19:35:48.090552 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 19:35:48.091862 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.084788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:45305 (local address 127.29.233.130:40643)
0810 19:35:48.085506 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.085511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.085532 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:35:48.085596 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.085599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.085660 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.085740 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:48.086367 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.087580 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.089752 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.091006 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.091201 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.091269 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.091582 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.091683 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526}
I20260810 19:35:48.092069 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.084132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:40643 (local address 127.29.233.131:45305)
0810 19:35:48.085335 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:48.085356 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:48.085497 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:48.085783 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:48.085788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:48.085813 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:48.086158 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.086169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.087760 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.087764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.089566 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.089582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.091212 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.091218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.091391 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.091443 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 19:35:48.091543 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 19:35:48.091672 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":3,"thread_start_us":822,"threads_started":1}
I20260810 19:35:48.091980 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d1cd034bc747909c7e9619cf03b7f4" candidate_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" is_pre_election: true
I20260810 19:35:48.092514 15867 raft_consensus.cc:482] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:35:48.092862 15867 raft_consensus.cc:504] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:48.093101 15639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d1cd034bc747909c7e9619cf03b7f4" candidate_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170056b505eb4a0995cec77493d3fbd2" is_pre_election: true
I20260810 19:35:48.093591 15639 raft_consensus.cc:2454] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3ba879ea9d4d89aaf7d0b0f626d0f9 in term 0.
I20260810 19:35:48.094063 15495 raft_consensus.cc:2454] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3ba879ea9d4d89aaf7d0b0f626d0f9 in term 0.
I20260810 19:35:48.094283 15715 leader_election.cc:302] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170056b505eb4a0995cec77493d3fbd2, 2c3ba879ea9d4d89aaf7d0b0f626d0f9; no voters:
I20260810 19:35:48.095469 15865 raft_consensus.cc:2781] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:35:48.095619 15865 raft_consensus.cc:482] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:35:48.095746 15865 raft_consensus.cc:3037] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Advancing to term 1
W20260810 19:35:48.096544 15546 tablet.cc:2369] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:35:48.098657 15865 raft_consensus.cc:504] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:48.098809 15867 leader_election.cc:288] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131:37729), 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643)
I20260810 19:35:48.099843 15495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d1cd034bc747909c7e9619cf03b7f4" candidate_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1"
I20260810 19:35:48.099918 15639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d1cd034bc747909c7e9619cf03b7f4" candidate_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170056b505eb4a0995cec77493d3fbd2"
I20260810 19:35:48.100070 15495 raft_consensus.cc:3037] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:48.100155 15639 raft_consensus.cc:3037] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:35:48.103572 15639 raft_consensus.cc:2454] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3ba879ea9d4d89aaf7d0b0f626d0f9 in term 1.
I20260810 19:35:48.103778 15495 raft_consensus.cc:2454] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3ba879ea9d4d89aaf7d0b0f626d0f9 in term 1.
I20260810 19:35:48.104154 15715 leader_election.cc:302] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170056b505eb4a0995cec77493d3fbd2, 2c3ba879ea9d4d89aaf7d0b0f626d0f9; no voters:
I20260810 19:35:48.104825 15865 leader_election.cc:288] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [CANDIDATE]: Term 1 election: Requested vote from peers b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129:39027), 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643)
I20260810 19:35:48.104969 15871 raft_consensus.cc:2781] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:35:48.105170 15871 raft_consensus.cc:686] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 LEADER]: Becoming Leader. State: Replica: 2c3ba879ea9d4d89aaf7d0b0f626d0f9, State: Running, Role: LEADER
I20260810 19:35:48.105834 15871 consensus_queue.cc:237] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } }
I20260810 19:35:48.106665 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.095004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:52729 (local address 127.29.233.131:37729)
0810 19:35:48.095211 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.095220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.098508 (+ 3288us) server_negotiation.cc:407] Connection header received
0810 19:35:48.098757 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.098762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.098841 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.098906 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:48.099790 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.100899 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.103937 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.105576 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.105758 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.105844 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.106205 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.106381 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":227,"server-negotiator.queue_time_us":56}
I20260810 19:35:48.106746 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.096348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:37729 (local address 127.29.233.129:52729)
0810 19:35:48.098419 (+ 2071us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:48.098438 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:48.098604 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:48.099122 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:48.099126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:48.099145 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:48.099583 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.099594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.101112 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.101117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.103738 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.103756 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.105811 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.105820 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.106000 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.106047 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 19:35:48.106167 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 19:35:48.106329 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1878,"mutex_wait_us":54,"thread_start_us":1479,"threads_started":1}
I20260810 19:35:48.107633 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.095532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53367 (local address 127.29.233.130:40643)
0810 19:35:48.095985 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.095993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.098339 (+ 2346us) server_negotiation.cc:407] Connection header received
0810 19:35:48.098649 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.098655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.098733 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.098799 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:35:48.099988 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.101315 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.104418 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.105712 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.105844 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.105907 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.107252 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.107398 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260810 19:35:48.107618 15788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d1cd034bc747909c7e9619cf03b7f4" candidate_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" is_pre_election: true
I20260810 19:35:48.107905 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.094961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:40643 (local address 127.29.233.129:53367)
0810 19:35:48.098213 (+ 3252us) negotiation.cc:107] Waiting for socket to connect
0810 19:35:48.098235 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:35:48.098468 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:35:48.099208 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:35:48.099215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:35:48.099261 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:35:48.099801 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.099829 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.101511 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.101516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.104161 (+ 2645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:35:48.104177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.106169 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.106175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:35:48.107061 (+ 886us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.107095 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 19:35:48.107205 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 19:35:48.107341 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3108,"mutex_wait_us":31,"spinlock_wait_cycles":10624,"thread_start_us":3091,"threads_started":1}
I20260810 19:35:48.108450 15639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d1cd034bc747909c7e9619cf03b7f4" candidate_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170056b505eb4a0995cec77493d3fbd2" is_pre_election: true
I20260810 19:35:48.108769 15639 raft_consensus.cc:2379] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7981fec0ee443c3a42ffa76d3f60ca1 in current term 1: Already voted for candidate 2c3ba879ea9d4d89aaf7d0b0f626d0f9 in this term.
I20260810 19:35:48.109411 15426 leader_election.cc:302] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7981fec0ee443c3a42ffa76d3f60ca1; no voters: 170056b505eb4a0995cec77493d3fbd2, 2c3ba879ea9d4d89aaf7d0b0f626d0f9
I20260810 19:35:48.109850 15867 raft_consensus.cc:2726] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 19:35:48.111682 15338 catalog_manager.cc:5495] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131). New cstate: current_term: 1 leader_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } health_report { overall_health: UNKNOWN } } }
I20260810 19:35:48.135097 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.129161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39024 (local address 127.29.233.131:37729)
0810 19:35:48.129367 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.129373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.129500 (+ 127us) server_negotiation.cc:407] Connection header received
0810 19:35:48.129770 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.129774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.129868 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.130021 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:35:48.130517 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.131316 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.132782 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.133197 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.133325 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.133862 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:35:48.134459 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:35:48.134628 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.134710 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.134795 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:35:48.143445 15495 raft_consensus.cc:1261] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Refusing update from remote peer 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:48.143446 15639 raft_consensus.cc:1261] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Refusing update from remote peer 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:35:48.144448 15865 consensus_queue.cc:1035] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:48.144747 15871 consensus_queue.cc:1035] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:35:48.158694 15874 mvcc.cc:205] Tried to move back new op lower bound from 7317055685185187840 to 7317055685047599104. Current Snapshot: MvccSnapshot[applied={T|T < 7317055685185187840}]
I20260810 19:35:48.160323 15875 mvcc.cc:205] Tried to move back new op lower bound from 7317055685185187840 to 7317055685047599104. Current Snapshot: MvccSnapshot[applied={T|T < 7317055685185187840}]
I20260810 19:35:48.173971 15338 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:48.174280 15338 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:35:48.179643 15338 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Sending DeleteTablet for 3 replicas of tablet 17d1cd034bc747909c7e9619cf03b7f4
I20260810 19:35:48.181206 15475 tablet_service.cc:1514] Processing DeleteTablet for tablet 17d1cd034bc747909c7e9619cf03b7f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:48 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:47730
I20260810 19:35:48.181214 15767 tablet_service.cc:1514] Processing DeleteTablet for tablet 17d1cd034bc747909c7e9619cf03b7f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:48 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:39022
I20260810 19:35:48.181614 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 15406
I20260810 19:35:48.182175 15474 tablet_service.cc:1514] Processing DeleteTablet for tablet 17d1cd034bc747909c7e9619cf03b7f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:48 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:47730
I20260810 19:35:48.182394 15619 tablet_service.cc:1514] Processing DeleteTablet for tablet 17d1cd034bc747909c7e9619cf03b7f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:35:48 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:36148
I20260810 19:35:48.210593 15885 tablet_replica.cc:330] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: stopping tablet replica
I20260810 19:35:48.211270 15886 tablet_replica.cc:330] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: stopping tablet replica
I20260810 19:35:48.211239 15885 raft_consensus.cc:2229] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:48.211683 15886 raft_consensus.cc:2229] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:35:48.212049 15885 raft_consensus.cc:2258] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:48.212423 15886 raft_consensus.cc:2258] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:48.213310 15888 tablet_replica.cc:330] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: stopping tablet replica
I20260810 19:35:48.213742 15888 raft_consensus.cc:2229] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:35:48.214788 15888 raft_consensus.cc:2258] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:35:48.215499 15886 ts_tablet_manager.cc:1905] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:48.216358 15885 ts_tablet_manager.cc:1905] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:48.219581 15888 ts_tablet_manager.cc:1905] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:35:48.223395 15886 ts_tablet_manager.cc:1918] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:48.223634 15886 log.cc:1196] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/17d1cd034bc747909c7e9619cf03b7f4
Aug 10 19:35:48 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:35:48.223934 15885 ts_tablet_manager.cc:1918] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:48.224095 15885 log.cc:1196] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/17d1cd034bc747909c7e9619cf03b7f4
I20260810 19:35:48.224356 15886 ts_tablet_manager.cc:1939] T 17d1cd034bc747909c7e9619cf03b7f4 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Deleting consensus metadata
I20260810 19:35:48.224643 15885 ts_tablet_manager.cc:1939] T 17d1cd034bc747909c7e9619cf03b7f4 P b7981fec0ee443c3a42ffa76d3f60ca1: Deleting consensus metadata
I20260810 19:35:48.226486 15326 catalog_manager.cc:4841] TS b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129:39027): tablet 17d1cd034bc747909c7e9619cf03b7f4 (table test-table [id=320c54851d774e899eb728858d989000]) successfully deleted
I20260810 19:35:48.226835 15326 catalog_manager.cc:4841] TS 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131:37729): tablet 17d1cd034bc747909c7e9619cf03b7f4 (table test-table [id=320c54851d774e899eb728858d989000]) successfully deleted
W20260810 19:35:48.227253 15326 catalog_manager.cc:4808] TS b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129:39027): delete failed for tablet 17d1cd034bc747909c7e9619cf03b7f4 because the tablet was not found. No further retry: Not found: Tablet not found: 17d1cd034bc747909c7e9619cf03b7f4
I20260810 19:35:48.232692 15888 ts_tablet_manager.cc:1918] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:35:48.232877 15888 log.cc:1196] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/17d1cd034bc747909c7e9619cf03b7f4
I20260810 19:35:48.233351 15888 ts_tablet_manager.cc:1939] T 17d1cd034bc747909c7e9619cf03b7f4 P 170056b505eb4a0995cec77493d3fbd2: Deleting consensus metadata
I20260810 19:35:48.235360 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.182920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.29.233.129:39027)
0810 19:35:48.183142 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.183153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.206155 (+ 23002us) server_negotiation.cc:407] Connection header received
0810 19:35:48.206852 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.206861 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.206982 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.207080 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:48.212277 (+ 5197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.213480 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.216046 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.216714 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.216822 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.227689 (+ 10867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:48.227704 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:48.227722 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:48.227770 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:48.231710 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:48.232732 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:48.232742 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:48.232745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:48.232935 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:48.233451 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:48.233456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:48.233462 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:48.233981 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:48.234340 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.234714 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.234978 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:35:48.235637 15324 catalog_manager.cc:4841] TS 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643): tablet 17d1cd034bc747909c7e9619cf03b7f4 (table test-table [id=320c54851d774e899eb728858d989000]) successfully deleted
I20260810 19:35:48.236096 15538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:48.330978 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 15406
I20260810 19:35:48.353338 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 15551
Aug 10 19:35:48 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:35:48.372660 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.354235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.29.233.130:40643)
0810 19:35:48.354451 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.354456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.354632 (+ 176us) server_negotiation.cc:407] Connection header received
0810 19:35:48.354782 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.354789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.354878 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.354995 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:48.356414 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.357558 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.359189 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.359844 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.359987 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.364813 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:48.364833 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:48.364856 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:48.364912 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:48.369194 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:48.370014 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:48.370025 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:48.370040 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:48.370236 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:48.370764 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:48.370769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:48.370773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:48.371287 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:48.371651 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.371960 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.372235 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:35:48.373473 15685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:48.463754 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 15551
I20260810 19:35:48.485733 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 15696
Aug 10 19:35:48 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:35:48.505877 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:35:48.486782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.29.233.131:37729)
0810 19:35:48.487054 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 19:35:48.487066 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:35:48.487134 (+ 68us) server_negotiation.cc:407] Connection header received
0810 19:35:48.487367 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:35:48.487371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:35:48.487434 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:35:48.487554 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:35:48.489620 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.490616 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.492619 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:35:48.493111 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:35:48.493211 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:35:48.499484 (+ 6273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:35:48.499500 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:35:48.499526 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:35:48.499563 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:35:48.502846 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:48.503710 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:48.503717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:48.503719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:48.503872 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:35:48.504316 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:35:48.504319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:35:48.504321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:35:48.504762 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:35:48.504996 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 19:35:48.505434 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 19:35:48.505606 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 19:35:48.506847 15830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:48.635178 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 15696
I20260810 19:35:48.659179 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 15307
I20260810 19:35:48.660434 15372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:35:48.774924 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 15307
I20260810 19:36:08.801614 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42273
--webserver_interface=127.29.233.190
--webserver_port=33557
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:42273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:09.156416 15893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:09.156949 15893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:09.157145 15893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:09.165709 15893 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 19:36:09.165844 15893 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 19:36:09.165915 15893 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 19:36:09.165989 15893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:36:09.166074 15893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:09.166178 15893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:09.166272 15893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:36:09.166373 15893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:36:09.176651 15893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:42273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=33557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:09.178057 15893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:09.179486 15893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:09.205525 15901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:09.205564 15904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:09.205588 15902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:09.205868 15893 server_base.cc:1034] running on GCE node
I20260810 19:36:09.207122 15893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:09.208829 15893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:09.210104 15893 hybrid_clock.cc:648] HybridClock initialized: now 1786390569210032 us; error 62 us; skew 500 ppm
I20260810 19:36:09.210566 15893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:09 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390569, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:09.214398 15893 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:36:09.216063 15893 webserver.cc:460] Webserver started at http://127.29.233.190:33557/ using document root <none> and password file <none>
I20260810 19:36:09.216915 15893 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:09.217061 15893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:09.223309 15893 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 19:36:09.226639 15911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:09.228072 15893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:36:09.228283 15893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "4d568c2817be415290e81e59c020a5e9"
format_stamp: "Formatted at 2026-08-10 19:35:45 on dist-test-slave-tm1g"
server_key: "88aad8a079effe6a096515101e9c192f"
server_key_iv: "ff4551a5b7d729d04f1e2e6dd5c989d5"
server_key_version: "encryptionkey@0"
I20260810 19:36:09.229408 15893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:09.247823 15893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:09.252612 15893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:09.252957 15893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:09.273854 15893 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:42273
I20260810 19:36:09.273911 15964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:42273 every 8 connection(s)
I20260810 19:36:09.275558 15893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:36:09.276450 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 15893
I20260810 19:36:09.277449 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:39027
--local_ip_for_outbound_sockets=127.29.233.129
--tserver_master_addrs=127.29.233.190:42273
--webserver_port=41905
--webserver_interface=127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:36:09.283200 15965 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:36:09.286773 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Bootstrap starting.
I20260810 19:36:09.308066 15965 log.cc:824] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:09.320480 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:36:09.321074 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Bootstrap complete.
I20260810 19:36:09.330662 15965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:09.331596 15965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d568c2817be415290e81e59c020a5e9, State: Initialized, Role: FOLLOWER
I20260810 19:36:09.332226 15965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:09.332487 15965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:36:09.332619 15965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:36:09.332823 15965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 1 FOLLOWER]: Advancing to term 2
I20260810 19:36:09.337095 15965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:09.337802 15965 leader_election.cc:302] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d568c2817be415290e81e59c020a5e9; no voters:
I20260810 19:36:09.338634 15965 leader_election.cc:288] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 19:36:09.338881 15970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 19:36:09.339993 15970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 LEADER]: Becoming Leader. State: Replica: 4d568c2817be415290e81e59c020a5e9, State: Running, Role: LEADER
I20260810 19:36:09.340718 15970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:09.340950 15965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:36:09.343683 15971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4d568c2817be415290e81e59c020a5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } } }
I20260810 19:36:09.343658 15972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d568c2817be415290e81e59c020a5e9. Latest consensus state: current_term: 2 leader_uuid: "4d568c2817be415290e81e59c020a5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } } }
I20260810 19:36:09.344321 15972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:09.344322 15971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:09.345366 15976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:36:09.364517 15976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:36:09.366062 15976 catalog_manager.cc:1259] Loaded cluster ID: 0ef9c54e8f694005860496fe78406f85
I20260810 19:36:09.366212 15976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:36:09.370461 15976 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:36:09.376073 15976 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Loaded TSK: 0
I20260810 19:36:09.419929 15976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Generated new TSK 1
I20260810 19:36:09.420490 15976 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 19:36:09.667186 15967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:09.667513 15967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:09.667701 15967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:09.674983 15967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:09.675200 15967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:09.675472 15967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:36:09.685631 15967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:39027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=41905
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:09.687125 15967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:09.688570 15967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:09.703222 15995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:09.703197 15997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:09.703814 15994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:09.704351 15967 server_base.cc:1034] running on GCE node
I20260810 19:36:09.705010 15967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:09.706051 15967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:09.707310 15967 hybrid_clock.cc:648] HybridClock initialized: now 1786390569707224 us; error 68 us; skew 500 ppm
I20260810 19:36:09.707652 15967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:09 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390569, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:09.711557 15967 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:36:09.713128 15967 webserver.cc:460] Webserver started at http://127.29.233.129:41905/ using document root <none> and password file <none>
I20260810 19:36:09.713825 15967 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:09.713989 15967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:09.719487 15967 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:36:09.722294 16004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:09.723773 15967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:36:09.723945 15967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "b7981fec0ee443c3a42ffa76d3f60ca1"
format_stamp: "Formatted at 2026-08-10 19:35:46 on dist-test-slave-tm1g"
server_key: "b06bc8c2ab1628cd2005bb15b231aa31"
server_key_iv: "799477e748c9dc39a09fecad1f0f309b"
server_key_version: "encryptionkey@0"
I20260810 19:36:09.724892 15967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:09.748886 15967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:09.752874 15967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:09.753171 15967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:09.754200 15967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:09.755584 15967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:09.755671 15967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:09.755827 15967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:09.755925 15967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:09.790529 15967 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39027
I20260810 19:36:09.790622 16124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39027 every 8 connection(s)
I20260810 19:36:09.791993 15967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:36:09.800287 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 15967
I20260810 19:36:09.801101 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:40643
--local_ip_for_outbound_sockets=127.29.233.130
--tserver_master_addrs=127.29.233.190:42273
--webserver_port=41693
--webserver_interface=127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:36:09 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390569, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:09.817587 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:09.794926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:58983 (local address 127.29.233.190:42273)
0810 19:36:09.795602 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:09.795622 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:09.796184 (+ 562us) server_negotiation.cc:407] Connection header received
0810 19:36:09.798340 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:09.798362 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:09.798694 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:09.799037 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:09.800948 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:09.802376 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:09.804833 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:09.805485 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:09.805644 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:09.809943 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:09.809951 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:09.809964 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:09.810004 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:09.812980 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:09.813731 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:09.813750 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:09.813756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:09.813912 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:09.814370 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:09.814377 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:09.814380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:09.814849 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:09.815066 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:09.815870 (+ 804us) server_negotiation.cc:299] Negotiation successful
0810 19:36:09.816250 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":267,"threads_started":1}
I20260810 19:36:09.817587 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:09.795158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:58983)
0810 19:36:09.795996 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:09.796043 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:09.797781 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:09.799191 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:09.799202 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:09.799961 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:09.800744 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:09.800770 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:09.802548 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:09.802557 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:09.804685 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:09.804703 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:09.805751 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:09.805760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:09.806084 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:09.806831 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:36:09.806872 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:09.809639 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:09.813134 (+ 3495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:09.813143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:09.813157 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:09.813591 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:09.814044 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:09.814049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:09.814053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:09.814236 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:09.815019 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:09.815027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:09.815359 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 19:36:09.815650 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 19:36:09.816213 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":354,"threads_started":1}
I20260810 19:36:09.820854 16125 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42273
I20260810 19:36:09.821213 16125 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:09.822093 16125 heartbeater.cc:511] Master 127.29.233.190:42273 requested a full tablet report, sending...
I20260810 19:36:09.824684 15929 ts_manager.cc:194] Registered new tserver with Master: b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129:39027)
I20260810 19:36:09.826997 15929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:58983
W20260810 19:36:10.153196 16129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:10.153486 16129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:10.153620 16129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:10.162194 16129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:10.162359 16129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:10.162585 16129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:36:10.172676 16129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:40643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=41693
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:10.174103 16129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:10.175482 16129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:10.188849 16135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:10.188850 16134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:10.189071 16138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:10.189642 16129 server_base.cc:1034] running on GCE node
I20260810 19:36:10.190335 16129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:10.191370 16129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:10.192598 16129 hybrid_clock.cc:648] HybridClock initialized: now 1786390570192554 us; error 34 us; skew 500 ppm
I20260810 19:36:10.192929 16129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:10 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:10.196105 16129 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:36:10.197469 16129 webserver.cc:460] Webserver started at http://127.29.233.130:41693/ using document root <none> and password file <none>
I20260810 19:36:10.198138 16129 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:10.198253 16129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:10.203954 16129 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 19:36:10.206748 16145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:10.208144 16129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:36:10.208305 16129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "170056b505eb4a0995cec77493d3fbd2"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "c32887a168936306bc5b399c500d5ced"
server_key_iv: "595c612198651f286ffd8f8f2e4a5926"
server_key_version: "encryptionkey@0"
I20260810 19:36:10.209265 16129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:10.229041 16129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:10.233664 16129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:10.233949 16129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:10.235087 16129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:10.236449 16129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:10.236575 16129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:10.236706 16129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:10.236801 16129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:10.271214 16129 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:40643
I20260810 19:36:10.271281 16265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:40643 every 8 connection(s)
I20260810 19:36:10.273023 16129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:36:10.276090 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 16129
I20260810 19:36:10.276863 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:37729
--local_ip_for_outbound_sockets=127.29.233.131
--tserver_master_addrs=127.29.233.190:42273
--webserver_port=33031
--webserver_interface=127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 19:36:10 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:10.299791 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.275816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:43977 (local address 127.29.233.190:42273)
0810 19:36:10.278500 (+ 2684us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.278506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.278531 (+ 25us) server_negotiation.cc:407] Connection header received
0810 19:36:10.280184 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.280190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.280287 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.280391 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:10.282584 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.283682 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.286828 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.287550 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.287671 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.292376 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:10.292385 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:10.292392 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:10.292447 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:10.296160 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.297058 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.297066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.297072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.297237 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.297695 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.297703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.297709 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.298126 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:10.298278 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:10.299072 (+ 794us) server_negotiation.cc:299] Negotiation successful
0810 19:36:10.299349 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2546}
I20260810 19:36:10.300442 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.276192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:43977)
0810 19:36:10.277101 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:10.277172 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:10.279854 (+ 2682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:10.280698 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:10.280712 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:10.281532 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:10.282398 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.282425 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.283851 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.283859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.286673 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.286692 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.287793 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.287803 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.288154 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.288979 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:36:10.289013 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:10.292092 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:10.296317 (+ 4225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:10.296324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:10.296338 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:10.296882 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:10.297357 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:10.297361 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:10.297363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:10.297566 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:10.298253 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:10.298259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:10.298651 (+ 392us) client_negotiation.cc:769] Sending connection context
0810 19:36:10.298891 (+ 240us) client_negotiation.cc:240] Negotiation successful
0810 19:36:10.299293 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":351,"threads_started":1}
I20260810 19:36:10.301986 16267 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42273
I20260810 19:36:10.302382 16267 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:10.303256 16267 heartbeater.cc:511] Master 127.29.233.190:42273 requested a full tablet report, sending...
I20260810 19:36:10.304860 15929 ts_manager.cc:194] Registered new tserver with Master: 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643)
I20260810 19:36:10.305701 15929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:43977
W20260810 19:36:10.633973 16270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:10.634348 16270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:10.634467 16270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:10.642063 16270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:10.642171 16270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:10.642284 16270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:36:10.652824 16270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:37729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=33031
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:10.654102 16270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:10.655483 16270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:10.669031 16276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:10.669072 16277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:10.669319 16279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:10.669826 16270 server_base.cc:1034] running on GCE node
I20260810 19:36:10.670516 16270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:10.671419 16270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:10.672616 16270 hybrid_clock.cc:648] HybridClock initialized: now 1786390570672576 us; error 31 us; skew 500 ppm
I20260810 19:36:10.672935 16270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:10 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:10.676136 16270 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:36:10.677618 16270 webserver.cc:460] Webserver started at http://127.29.233.131:33031/ using document root <none> and password file <none>
I20260810 19:36:10.678382 16270 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:10.678503 16270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:10.683872 16270 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:36:10.686764 16286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:10.688199 16270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:36:10.688362 16270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "d3efd9235f833bfe9a3e839d12995270"
server_key_iv: "4eb3eea796173991080ebb518791da4b"
server_key_version: "encryptionkey@0"
I20260810 19:36:10.689358 16270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:10.705749 16270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:10.710035 16270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:10.710381 16270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:10.711472 16270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:10.712884 16270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:10.712993 16270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:10.713143 16270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:10.713228 16270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:10.748070 16270 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:37729
I20260810 19:36:10.748162 16407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:37729 every 8 connection(s)
I20260810 19:36:10.749648 16270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:36:10.758476 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 16270
Aug 10 19:36:10 dist-test-slave-tm1g krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390570, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:10.771996 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.752458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:55455 (local address 127.29.233.190:42273)
0810 19:36:10.752650 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.752654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.753748 (+ 1094us) server_negotiation.cc:407] Connection header received
0810 19:36:10.755714 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.755720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.755831 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.755944 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:10.757449 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.758483 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.760630 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.761114 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.761209 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.765284 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:10.765306 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:10.765316 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:10.765356 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:10.768239 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.769210 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.769219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.769222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.769391 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.769822 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.769828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.769832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.770233 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:10.770438 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:10.771236 (+ 798us) server_negotiation.cc:299] Negotiation successful
0810 19:36:10.771552 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"spinlock_wait_cycles":1408}
I20260810 19:36:10.772725 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.752845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:55455)
0810 19:36:10.753589 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:10.753627 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:10.755440 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:10.756179 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:10.756187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:10.756702 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:10.757264 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.757280 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.758715 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.758720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.760473 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.760486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.761354 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.761362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.761692 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.762532 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:36:10.762567 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:10.765075 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:10.768395 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:10.768406 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:10.768426 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:10.769076 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:10.769509 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:10.769513 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:10.769515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:10.769701 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:10.770435 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:10.770442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:10.770776 (+ 334us) client_negotiation.cc:769] Sending connection context
0810 19:36:10.771110 (+ 334us) client_negotiation.cc:240] Negotiation successful
0810 19:36:10.771536 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":293,"threads_started":1}
I20260810 19:36:10.774483 16408 heartbeater.cc:347] Connected to a master server at 127.29.233.190:42273
I20260810 19:36:10.774796 16408 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:10.775518 16408 heartbeater.cc:511] Master 127.29.233.190:42273 requested a full tablet report, sending...
I20260810 19:36:10.777150 15929 ts_manager.cc:194] Registered new tserver with Master: 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131:37729)
I20260810 19:36:10.777282 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.759526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.29.233.190:42273)
0810 19:36:10.760347 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.760351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.760366 (+ 15us) server_negotiation.cc:407] Connection header received
0810 19:36:10.760406 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.760409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.760461 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.760554 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:10.762658 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.763394 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.765242 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.766033 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.766148 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.769320 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:10.769325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:10.769330 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:10.769381 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:10.772615 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.773506 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.773511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.773516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.773644 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.774232 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.774236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.774237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.774551 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:10.774858 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:10.776757 (+ 1899us) server_negotiation.cc:299] Negotiation successful
0810 19:36:10.776920 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"spinlock_wait_cycles":2432,"thread_start_us":606,"threads_started":1}
I20260810 19:36:10.778255 15929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:55455
I20260810 19:36:10.779955 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:36:10.786733 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.780907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.29.233.190:42273)
0810 19:36:10.781085 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.781089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.781590 (+ 501us) server_negotiation.cc:407] Connection header received
0810 19:36:10.781741 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.781744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.781858 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.781971 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:36:10.782518 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.783181 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.784801 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.785230 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.785335 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.785910 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:36:10.786156 (+ 246us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 19:36:10.786486 (+ 330us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42332: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 19:36:10.786859 16413 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42332: token expired
W20260810 19:36:10.787423 15840 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.233.190:42273)
I20260810 19:36:10.802529 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.787952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.29.233.190:42273)
0810 19:36:10.788109 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.788117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.788153 (+ 36us) server_negotiation.cc:407] Connection header received
0810 19:36:10.788336 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.788342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.788406 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.788512 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:10.790011 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.790802 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.792771 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.793224 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.793434 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.796529 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:10.796535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:10.796538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:10.796572 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:10.799480 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.800239 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.800247 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.800251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.800404 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:10.800820 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:10.800826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:10.800830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:10.801195 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:10.801483 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:10.801819 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 19:36:10.802083 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 19:36:10.805884 15929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:36:10.813328 15929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:36:10.831378 16125 heartbeater.cc:503] Master 127.29.233.190:42273 was elected leader, sending a full tablet report...
I20260810 19:36:10.846156 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.826357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.29.233.131:37729)
0810 19:36:10.828007 (+ 1650us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.828020 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.829853 (+ 1833us) server_negotiation.cc:407] Connection header received
0810 19:36:10.830341 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.830353 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.830683 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.830888 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:10.835874 (+ 4986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.839807 (+ 3933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.843077 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.844562 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.844718 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.844899 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:10.845647 (+ 748us) server_negotiation.cc:299] Negotiation successful
0810 19:36:10.845802 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1414,"thread_start_us":1214,"threads_started":1}
I20260810 19:36:10.846213 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.825654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:37729 (local address 127.0.0.1:52082)
0810 19:36:10.829679 (+ 4025us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:10.829734 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:10.830019 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:10.834953 (+ 4934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:10.834965 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:10.835011 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:10.835592 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.835608 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.839960 (+ 4352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.839969 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.842862 (+ 2893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.842878 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.845111 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.845117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.845364 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.845438 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 19:36:10.845575 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 19:36:10.845762 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3776,"mutex_wait_us":1,"thread_start_us":3621,"threads_started":1}
I20260810 19:36:10.847883 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.830780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:39027 (local address 127.0.0.1:46688)
0810 19:36:10.840490 (+ 9710us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:10.840511 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:10.840622 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:10.841222 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:10.841227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:10.841253 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:10.841756 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.841767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.843462 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.843467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.844899 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.844911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.847002 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.847008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.847177 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.847211 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 19:36:10.847298 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 19:36:10.847423 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9548,"mutex_wait_us":64,"thread_start_us":9376,"threads_started":1}
I20260810 19:36:10.848091 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.825350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:40643 (local address 127.0.0.1:35486)
0810 19:36:10.830280 (+ 4930us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:10.830341 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:10.830487 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:10.840593 (+ 10106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:10.840599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:10.840625 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:10.841068 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.841087 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.843105 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.843110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.845212 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:10.845224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.847232 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.847238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:10.847380 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.847403 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:36:10.847515 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 19:36:10.847652 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3191,"thread_start_us":2968,"threads_started":1}
I20260810 19:36:10.848728 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.830026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.29.233.129:39027)
0810 19:36:10.839639 (+ 9613us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.839648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.840759 (+ 1111us) server_negotiation.cc:407] Connection header received
0810 19:36:10.840866 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.840873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.841035 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.841225 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:10.841922 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.843292 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.845063 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.846685 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.846881 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.847055 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:10.848000 (+ 945us) server_negotiation.cc:299] Negotiation successful
0810 19:36:10.848162 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9387,"thread_start_us":9157,"threads_started":1}
I20260810 19:36:10.848858 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:10.825583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.29.233.130:40643)
0810 19:36:10.839910 (+ 14327us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:10.839918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:10.839960 (+ 42us) server_negotiation.cc:407] Connection header received
0810 19:36:10.840078 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:10.840087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:10.840374 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:10.840600 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:10.841271 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.842948 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.845426 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:10.846905 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:10.847071 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:10.847235 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:10.847579 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 19:36:10.848503 (+ 924us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14162,"thread_start_us":14019,"threads_started":1}
I20260810 19:36:10.850786 16337 tablet_service.cc:1467] Processing CreateTablet for tablet ff42e1ac203b4c6583b14e054439ab93 (DEFAULT_TABLE table=test-table [id=2d55561b377f4515896fa36cf34d6813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:10.852053 16337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff42e1ac203b4c6583b14e054439ab93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:10.852329 16198 tablet_service.cc:1467] Processing CreateTablet for tablet ff42e1ac203b4c6583b14e054439ab93 (DEFAULT_TABLE table=test-table [id=2d55561b377f4515896fa36cf34d6813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:10.852448 16058 tablet_service.cc:1467] Processing CreateTablet for tablet ff42e1ac203b4c6583b14e054439ab93 (DEFAULT_TABLE table=test-table [id=2d55561b377f4515896fa36cf34d6813]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:10.853658 16198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff42e1ac203b4c6583b14e054439ab93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:10.853672 16058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff42e1ac203b4c6583b14e054439ab93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:10.865516 16426 tablet_bootstrap.cc:492] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Bootstrap starting.
I20260810 19:36:10.868484 16426 tablet_bootstrap.cc:654] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:10.868927 16428 tablet_bootstrap.cc:492] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Bootstrap starting.
I20260810 19:36:10.870401 16426 log.cc:824] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:10.871893 16427 tablet_bootstrap.cc:492] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Bootstrap starting.
I20260810 19:36:10.872669 16428 tablet_bootstrap.cc:654] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:10.873787 16426 tablet_bootstrap.cc:492] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: No bootstrap required, opened a new log
I20260810 19:36:10.873971 16428 log.cc:824] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:10.874095 16426 ts_tablet_manager.cc:1397] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260810 19:36:10.875643 16427 tablet_bootstrap.cc:654] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:10.876921 16427 log.cc:824] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:10.877594 16428 tablet_bootstrap.cc:492] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: No bootstrap required, opened a new log
I20260810 19:36:10.877915 16428 ts_tablet_manager.cc:1397] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260810 19:36:10.879976 16427 tablet_bootstrap.cc:492] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: No bootstrap required, opened a new log
I20260810 19:36:10.880227 16427 ts_tablet_manager.cc:1397] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 19:36:10.882431 16426 raft_consensus.cc:348] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:10.882805 16426 raft_consensus.cc:374] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:10.882918 16426 raft_consensus.cc:729] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3ba879ea9d4d89aaf7d0b0f626d0f9, State: Initialized, Role: FOLLOWER
I20260810 19:36:10.883688 16426 consensus_queue.cc:260] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:10.884851 16408 heartbeater.cc:503] Master 127.29.233.190:42273 was elected leader, sending a full tablet report...
I20260810 19:36:10.885557 16426 ts_tablet_manager.cc:1428] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260810 19:36:10.888950 16428 raft_consensus.cc:348] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:10.889415 16428 raft_consensus.cc:374] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:10.889519 16428 raft_consensus.cc:729] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7981fec0ee443c3a42ffa76d3f60ca1, State: Initialized, Role: FOLLOWER
I20260810 19:36:10.890182 16428 consensus_queue.cc:260] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:10.891913 16428 ts_tablet_manager.cc:1428] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 19:36:10.893589 16427 raft_consensus.cc:348] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:10.894539 16427 raft_consensus.cc:374] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:10.894639 16427 raft_consensus.cc:729] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170056b505eb4a0995cec77493d3fbd2, State: Initialized, Role: FOLLOWER
I20260810 19:36:10.895258 16427 consensus_queue.cc:260] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:10.896420 16267 heartbeater.cc:503] Master 127.29.233.190:42273 was elected leader, sending a full tablet report...
I20260810 19:36:10.896886 16427 ts_tablet_manager.cc:1428] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
W20260810 19:36:11.001518 16409 tablet.cc:2369] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:36:11.025144 16268 tablet.cc:2369] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:36:11.044492 16126 tablet.cc:2369] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:36:11.046032 16434 raft_consensus.cc:482] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:36:11.046340 16434 raft_consensus.cc:504] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:11.048142 16434 leader_election.cc:288] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643), 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131:37729)
I20260810 19:36:11.055296 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.048594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:58007 (local address 127.29.233.130:40643)
0810 19:36:11.048891 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:11.048899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:11.049905 (+ 1006us) server_negotiation.cc:407] Connection header received
0810 19:36:11.050150 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:11.050155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:11.050215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:11.050337 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:11.050979 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.051732 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.053537 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.054292 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.054517 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.054599 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:11.054892 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 19:36:11.055016 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260810 19:36:11.055374 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.048640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:40643 (local address 127.29.233.129:58007)
0810 19:36:11.049772 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:11.049788 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:11.049977 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:11.050351 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:11.050356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:11.050379 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:11.050815 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:11.050838 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.051876 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.051887 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:11.053379 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:11.053392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.054561 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.054568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:11.054727 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.054751 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:36:11.054864 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 19:36:11.054997 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"thread_start_us":580,"threads_started":1}
I20260810 19:36:11.055361 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.048815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:42493 (local address 127.29.233.131:37729)
0810 19:36:11.049046 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:11.049054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:11.049979 (+ 925us) server_negotiation.cc:407] Connection header received
0810 19:36:11.050179 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:11.050183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:11.050241 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:11.050387 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:11.051059 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.051793 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.053597 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.054437 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.054625 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.054702 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:11.054969 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 19:36:11.055096 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:36:11.055429 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.048797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:37729 (local address 127.29.233.129:42493)
0810 19:36:11.049889 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:11.049903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:11.050019 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:11.050406 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:11.050410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:11.050437 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:11.050856 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:11.050865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.051941 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.051953 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:11.053436 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:11.053448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.054666 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.054673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:11.054817 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.054839 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 19:36:11.054944 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 19:36:11.055077 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":704,"threads_started":1}
I20260810 19:36:11.056432 16219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff42e1ac203b4c6583b14e054439ab93" candidate_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170056b505eb4a0995cec77493d3fbd2" is_pre_election: true
I20260810 19:36:11.056445 16358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff42e1ac203b4c6583b14e054439ab93" candidate_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" is_pre_election: true
I20260810 19:36:11.056921 16358 raft_consensus.cc:2454] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7981fec0ee443c3a42ffa76d3f60ca1 in term 0.
I20260810 19:36:11.056921 16219 raft_consensus.cc:2454] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7981fec0ee443c3a42ffa76d3f60ca1 in term 0.
I20260810 19:36:11.057588 16009 leader_election.cc:302] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3ba879ea9d4d89aaf7d0b0f626d0f9, b7981fec0ee443c3a42ffa76d3f60ca1; no voters:
I20260810 19:36:11.058012 16434 raft_consensus.cc:2781] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:36:11.058157 16434 raft_consensus.cc:482] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:36:11.058284 16434 raft_consensus.cc:3037] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:11.060635 16434 raft_consensus.cc:504] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:11.061303 16434 leader_election.cc:288] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [CANDIDATE]: Term 1 election: Requested vote from peers 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643), 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131:37729)
I20260810 19:36:11.061879 16358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff42e1ac203b4c6583b14e054439ab93" candidate_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9"
I20260810 19:36:11.061873 16219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff42e1ac203b4c6583b14e054439ab93" candidate_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170056b505eb4a0995cec77493d3fbd2"
I20260810 19:36:11.062103 16358 raft_consensus.cc:3037] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:11.062103 16219 raft_consensus.cc:3037] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:11.065493 16219 raft_consensus.cc:2454] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7981fec0ee443c3a42ffa76d3f60ca1 in term 1.
I20260810 19:36:11.065493 16358 raft_consensus.cc:2454] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7981fec0ee443c3a42ffa76d3f60ca1 in term 1.
I20260810 19:36:11.065972 16009 leader_election.cc:302] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 170056b505eb4a0995cec77493d3fbd2, 2c3ba879ea9d4d89aaf7d0b0f626d0f9, b7981fec0ee443c3a42ffa76d3f60ca1; no voters:
I20260810 19:36:11.066371 16434 raft_consensus.cc:2781] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:36:11.066910 16434 raft_consensus.cc:686] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 LEADER]: Becoming Leader. State: Replica: b7981fec0ee443c3a42ffa76d3f60ca1, State: Running, Role: LEADER
I20260810 19:36:11.067422 16434 consensus_queue.cc:237] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } }
I20260810 19:36:11.072326 15928 catalog_manager.cc:5495] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7981fec0ee443c3a42ffa76d3f60ca1" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 39027 } health_report { overall_health: HEALTHY } } }
I20260810 19:36:11.132520 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.126974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.29.233.129:39027)
0810 19:36:11.127194 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:11.127201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:11.127222 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:36:11.127461 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:11.127465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:11.127547 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:11.127704 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:36:11.128082 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.128897 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.130354 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.130761 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.130898 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.131421 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:36:11.131930 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:36:11.132106 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:11.132171 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 19:36:11.132253 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:36:11.141011 16219 raft_consensus.cc:1261] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Refusing update from remote peer b7981fec0ee443c3a42ffa76d3f60ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:36:11.141013 16358 raft_consensus.cc:1261] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 FOLLOWER]: Refusing update from remote peer b7981fec0ee443c3a42ffa76d3f60ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:36:11.142164 16439 consensus_queue.cc:1035] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170056b505eb4a0995cec77493d3fbd2" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:36:11.142652 16434 consensus_queue.cc:1035] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:36:11.154062 16442 mvcc.cc:205] Tried to move back new op lower bound from 7317055779382829056 to 7317055779096068096. Current Snapshot: MvccSnapshot[applied={T|T < 7317055779382829056}]
I20260810 19:36:11.168953 15929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:36:11.169269 15929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:36:11.173003 15929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Sending DeleteTablet for 3 replicas of tablet ff42e1ac203b4c6583b14e054439ab93
I20260810 19:36:11.174413 16337 tablet_service.cc:1514] Processing DeleteTablet for tablet ff42e1ac203b4c6583b14e054439ab93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:11 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:52082
I20260810 19:36:11.174482 16198 tablet_service.cc:1514] Processing DeleteTablet for tablet ff42e1ac203b4c6583b14e054439ab93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:11 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:35486
I20260810 19:36:11.174692 16058 tablet_service.cc:1514] Processing DeleteTablet for tablet ff42e1ac203b4c6583b14e054439ab93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:11 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:46688
I20260810 19:36:11.175446 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 15967
I20260810 19:36:11.186058 16452 tablet_replica.cc:330] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: stopping tablet replica
I20260810 19:36:11.186286 16454 tablet_replica.cc:330] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: stopping tablet replica
I20260810 19:36:11.186839 16454 raft_consensus.cc:2229] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:36:11.187585 16454 raft_consensus.cc:2258] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:11.188870 16452 raft_consensus.cc:2229] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:36:11.189432 16452 raft_consensus.cc:2258] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:11.189697 16454 ts_tablet_manager.cc:1905] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:11.190387 16453 tablet_replica.cc:330] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: stopping tablet replica
I20260810 19:36:11.190732 16453 raft_consensus.cc:2229] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:36:11.191216 16453 raft_consensus.cc:2258] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:11.193801 16453 ts_tablet_manager.cc:1905] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:11.194422 16452 ts_tablet_manager.cc:1905] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:11.196928 16454 ts_tablet_manager.cc:1918] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:11.197160 16454 log.cc:1196] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/ff42e1ac203b4c6583b14e054439ab93
I20260810 19:36:11.197741 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.176436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.29.233.129:39027)
0810 19:36:11.176656 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:11.176666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:11.176831 (+ 165us) server_negotiation.cc:407] Connection header received
0810 19:36:11.177062 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:11.177070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:11.177229 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:11.177364 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:11.179578 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.181146 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.183712 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.184677 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.184834 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.190049 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:11.190061 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:11.190077 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:11.190128 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:11.194468 (+ 4340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:11.195306 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:11.195317 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:11.195325 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:11.195528 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:11.195965 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:11.195970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:11.195974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:11.196389 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:11.196706 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:11.197061 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 19:36:11.197312 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:36:11.197821 16454 ts_tablet_manager.cc:1939] T ff42e1ac203b4c6583b14e054439ab93 P b7981fec0ee443c3a42ffa76d3f60ca1: Deleting consensus metadata
I20260810 19:36:11.198776 16120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:11.201473 16453 ts_tablet_manager.cc:1918] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:11.201638 16453 log.cc:1196] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/ff42e1ac203b4c6583b14e054439ab93
I20260810 19:36:11.202201 16453 ts_tablet_manager.cc:1939] T ff42e1ac203b4c6583b14e054439ab93 P 170056b505eb4a0995cec77493d3fbd2: Deleting consensus metadata
I20260810 19:36:11.204107 15913 catalog_manager.cc:4841] TS 170056b505eb4a0995cec77493d3fbd2 (127.29.233.130:40643): tablet ff42e1ac203b4c6583b14e054439ab93 (table test-table [id=2d55561b377f4515896fa36cf34d6813]) successfully deleted
I20260810 19:36:11.204267 16452 ts_tablet_manager.cc:1918] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:11.204449 16452 log.cc:1196] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/ff42e1ac203b4c6583b14e054439ab93
I20260810 19:36:11.204972 16452 ts_tablet_manager.cc:1939] T ff42e1ac203b4c6583b14e054439ab93 P 2c3ba879ea9d4d89aaf7d0b0f626d0f9: Deleting consensus metadata
I20260810 19:36:11.206463 15915 catalog_manager.cc:4841] TS 2c3ba879ea9d4d89aaf7d0b0f626d0f9 (127.29.233.131:37729): tablet ff42e1ac203b4c6583b14e054439ab93 (table test-table [id=2d55561b377f4515896fa36cf34d6813]) successfully deleted
I20260810 19:36:11.314152 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 15967
I20260810 19:36:11.314282 15915 catalog_manager.cc:4841] TS b7981fec0ee443c3a42ffa76d3f60ca1 (127.29.233.129:39027): tablet ff42e1ac203b4c6583b14e054439ab93 (table test-table [id=2d55561b377f4515896fa36cf34d6813]) successfully deleted
I20260810 19:36:11.335615 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 16129
I20260810 19:36:11.349977 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.336485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35502 (local address 127.29.233.130:40643)
0810 19:36:11.336689 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:11.336694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:11.336749 (+ 55us) server_negotiation.cc:407] Connection header received
0810 19:36:11.336922 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:11.336926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:11.336989 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:11.337081 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:11.338612 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.339382 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.340664 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.341143 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.341256 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.344193 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:11.344209 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:11.344226 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:11.344264 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:11.347154 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:11.347912 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:11.347920 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:11.347922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:11.348063 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:11.348454 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:11.348457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:11.348459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:11.348901 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:11.349126 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:11.349438 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 19:36:11.349606 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 19:36:11.350733 16261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:11.443512 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 16129
I20260810 19:36:11.464211 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 16270
I20260810 19:36:11.477592 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:11.465122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.29.233.131:37729)
0810 19:36:11.465361 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:11.465365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:11.465386 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:36:11.465455 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:11.465459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:11.465531 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:11.465619 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:11.467042 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.467780 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.469053 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:11.469473 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:11.469579 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:11.472272 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:11.472288 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:11.472306 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:11.472366 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:11.475072 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:11.475740 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:11.475747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:11.475750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:11.475905 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:11.476221 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:11.476224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:11.476226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:11.476564 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:11.476775 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:11.477062 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 19:36:11.477218 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 19:36:11.478241 16402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:11.567765 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 16270
I20260810 19:36:11.587611 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 15893
I20260810 19:36:11.588778 15960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:11.705461 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 15893
I20260810 19:36:31.732195 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42273
--webserver_interface=127.29.233.190
--webserver_port=33557
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:42273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:32.080632 16459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:32.080945 16459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:32.081037 16459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:32.088291 16459 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 19:36:32.088398 16459 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 19:36:32.088436 16459 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 19:36:32.088481 16459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:36:32.088534 16459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:32.088603 16459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:32.088649 16459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:36:32.088696 16459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:36:32.098417 16459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:42273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:42273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=33557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:32.099653 16459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:32.100935 16459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:32.113365 16464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:32.113338 16465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:32.113389 16467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:32.114059 16459 server_base.cc:1034] running on GCE node
I20260810 19:36:32.115075 16459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:32.116547 16459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:32.117823 16459 hybrid_clock.cc:648] HybridClock initialized: now 1786390592117758 us; error 37 us; skew 500 ppm
I20260810 19:36:32.118191 16459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:32 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:32.121392 16459 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:36:32.122905 16459 webserver.cc:460] Webserver started at http://127.29.233.190:33557/ using document root <none> and password file <none>
I20260810 19:36:32.123634 16459 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:32.123775 16459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:32.129469 16459 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 19:36:32.132534 16474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:32.133854 16459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:36:32.134063 16459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "4d568c2817be415290e81e59c020a5e9"
format_stamp: "Formatted at 2026-08-10 19:35:45 on dist-test-slave-tm1g"
server_key: "88aad8a079effe6a096515101e9c192f"
server_key_iv: "ff4551a5b7d729d04f1e2e6dd5c989d5"
server_key_version: "encryptionkey@0"
I20260810 19:36:32.135121 16459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:32.154367 16459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:32.158522 16459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:32.158834 16459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:32.177847 16459 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:42273
I20260810 19:36:32.177893 16532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:42273 every 8 connection(s)
I20260810 19:36:32.179445 16459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:36:32.186481 16533 sys_catalog.cc:263] Verifying existing consensus state
I20260810 19:36:32.186775 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 16459
I20260810 19:36:32.187654 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:39027
--local_ip_for_outbound_sockets=127.29.233.129
--tserver_master_addrs=127.29.233.190:42273
--webserver_port=41905
--webserver_interface=127.29.233.129
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:36:32.189818 16533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Bootstrap starting.
I20260810 19:36:32.213965 16533 log.cc:824] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:32.236969 16533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 19:36:32.237684 16533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Bootstrap complete.
I20260810 19:36:32.249902 16533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:32.250929 16533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d568c2817be415290e81e59c020a5e9, State: Initialized, Role: FOLLOWER
I20260810 19:36:32.251653 16533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:32.251907 16533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:36:32.252027 16533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:36:32.252226 16533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 2 FOLLOWER]: Advancing to term 3
I20260810 19:36:32.256160 16533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:32.256673 16533 leader_election.cc:302] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d568c2817be415290e81e59c020a5e9; no voters:
I20260810 19:36:32.257390 16533 leader_election.cc:288] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 19:36:32.257539 16537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 19:36:32.258689 16537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [term 3 LEADER]: Becoming Leader. State: Replica: 4d568c2817be415290e81e59c020a5e9, State: Running, Role: LEADER
I20260810 19:36:32.259284 16537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } }
I20260810 19:36:32.259735 16533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:36:32.262472 16538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4d568c2817be415290e81e59c020a5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } } }
I20260810 19:36:32.262861 16538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:32.262032 16539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d568c2817be415290e81e59c020a5e9. Latest consensus state: current_term: 3 leader_uuid: "4d568c2817be415290e81e59c020a5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d568c2817be415290e81e59c020a5e9" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 42273 } } }
I20260810 19:36:32.263525 16539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9 [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:32.268596 16543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:36:32.279649 16543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:36:32.281618 16543 catalog_manager.cc:1259] Loaded cluster ID: 0ef9c54e8f694005860496fe78406f85
I20260810 19:36:32.281749 16543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:36:32.287355 16543 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:36:32.289736 16543 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Loaded TSK: 0
I20260810 19:36:32.289917 16543 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d568c2817be415290e81e59c020a5e9: Loaded TSK: 1
F20260810 19:36:32.291383 16543 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f24f24909ad google::LogMessage::Fail() at ??:0
@ 0x7f24f249388a google::LogMessage::SendToLog() at ??:0
@ 0x7f24f24904de google::LogMessage::Flush() at ??:0
@ 0x7f24f2491a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260810 19:36:32.303611 16556 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f24f61f47d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f24f61e373e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f2507ebf1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2507ebe783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f2507f4a2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f2507f4a0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f2507f945e9 std::function<>::operator()() at ??:0
@ 0x7f2507ec2cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f2507ec0c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2507f48610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f2507f48439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f24f50dc345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f24f50df91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f24f50df799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f24f50baeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f24f83946db start_thread at ??:0
@ 0x7f24f132a61f clone at ??:0
W20260810 19:36:32.599022 16535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:32.599381 16535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:32.599567 16535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:32.606935 16535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:32.607087 16535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:32.607267 16535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:36:32.617125 16535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:39027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=41905
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:32.618553 16535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:32.619936 16535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:32.633725 16585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:32.633937 16582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:32.633874 16583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:32.634863 16535 server_base.cc:1034] running on GCE node
I20260810 19:36:32.635524 16535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:32.636616 16535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:32.637852 16535 hybrid_clock.cc:648] HybridClock initialized: now 1786390592637787 us; error 60 us; skew 500 ppm
I20260810 19:36:32.638199 16535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:32 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:32.647053 16535 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:36:32.648728 16535 webserver.cc:460] Webserver started at http://127.29.233.129:41905/ using document root <none> and password file <none>
I20260810 19:36:32.649534 16535 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:32.649700 16535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:32.655700 16535 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 19:36:32.658736 16592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:32.660162 16535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:36:32.660360 16535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "b7981fec0ee443c3a42ffa76d3f60ca1"
format_stamp: "Formatted at 2026-08-10 19:35:46 on dist-test-slave-tm1g"
server_key: "b06bc8c2ab1628cd2005bb15b231aa31"
server_key_iv: "799477e748c9dc39a09fecad1f0f309b"
server_key_version: "encryptionkey@0"
I20260810 19:36:32.661405 16535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:32.677832 16535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:32.681921 16535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:32.682214 16535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:32.683305 16535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:32.684710 16535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:32.684839 16535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:32.684974 16535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:32.685068 16535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:32.720530 16535 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:39027
I20260810 19:36:32.720621 16712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:39027 every 8 connection(s)
I20260810 19:36:32.722091 16535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:36:32.726994 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:32.725081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:40799)
0810 19:36:32.725832 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:32.726059 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":306,"threads_started":1}
W20260810 19:36:32.727622 16713 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:42273 (0 consecutive failures): Network error: Failed to ping master at 127.29.233.190:42273: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
I20260810 19:36:32.728012 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 16535
I20260810 19:36:32.728478 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:32.728168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:43301)
0810 19:36:32.728302 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:32.728377 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 19:36:32.728734 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:40643
--local_ip_for_outbound_sockets=127.29.233.130
--tserver_master_addrs=127.29.233.190:42273
--webserver_port=41693
--webserver_interface=127.29.233.130
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:36:32.729314 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:32.728989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:48235)
0810 19:36:32.729127 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:32.729207 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 19:36:32.729583 16713 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 19:36:33.083019 16716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:33.083374 16716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:33.083557 16716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:33.091092 16716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:33.091250 16716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:33.091435 16716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:36:33.103519 16716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:40643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=41693
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:33.104945 16716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:33.106406 16716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:33.120342 16724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:33.120391 16726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:33.120443 16721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:33.120925 16716 server_base.cc:1034] running on GCE node
I20260810 19:36:33.121695 16716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:33.122747 16716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:33.123999 16716 hybrid_clock.cc:648] HybridClock initialized: now 1786390593123937 us; error 54 us; skew 500 ppm
I20260810 19:36:33.124333 16716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:33 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390593, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:33.127547 16716 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:36:33.129026 16716 webserver.cc:460] Webserver started at http://127.29.233.130:41693/ using document root <none> and password file <none>
I20260810 19:36:33.129753 16716 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:33.129908 16716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:33.135535 16716 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 19:36:33.138635 16734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:33.140182 16716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:36:33.140394 16716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "170056b505eb4a0995cec77493d3fbd2"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "c32887a168936306bc5b399c500d5ced"
server_key_iv: "595c612198651f286ffd8f8f2e4a5926"
server_key_version: "encryptionkey@0"
I20260810 19:36:33.141456 16716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:33.154736 16716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:33.159462 16716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:33.159793 16716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:33.160782 16716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:33.162115 16716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:33.162214 16716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:33.162352 16716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:33.162469 16716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 19:36:33.195623 16716 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:40643
I20260810 19:36:33.195713 16855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:40643 every 8 connection(s)
I20260810 19:36:33.197121 16716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:36:33.200901 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 16716
I20260810 19:36:33.201692 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:37729
--local_ip_for_outbound_sockets=127.29.233.131
--tserver_master_addrs=127.29.233.190:42273
--webserver_port=33031
--webserver_interface=127.29.233.131
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:36:33.202268 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:33.200171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:38473)
0810 19:36:33.201045 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:33.201311 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":393,"threads_started":1}
W20260810 19:36:33.203008 16856 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:42273 (0 consecutive failures): Network error: Failed to ping master at 127.29.233.190:42273: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
I20260810 19:36:33.203918 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:33.203596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:40421)
0810 19:36:33.203747 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:33.203815 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 19:36:33.204912 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:33.204465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:40727)
0810 19:36:33.204642 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:33.204745 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 19:36:33.205229 16856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 19:36:33.549057 16859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:33.549353 16859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:33.549525 16859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:33.556936 16859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:33.557058 16859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:33.557247 16859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:36:33.567473 16859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:37729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=33031
--tserver_master_addrs=127.29.233.190:42273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:33.568907 16859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:33.570272 16859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:33.583329 16864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:33.583395 16865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:33.583601 16867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:33.584520 16859 server_base.cc:1034] running on GCE node
I20260810 19:36:33.585187 16859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:33.586261 16859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:33.587507 16859 hybrid_clock.cc:648] HybridClock initialized: now 1786390593587458 us; error 49 us; skew 500 ppm
I20260810 19:36:33.587785 16859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:33 dist-test-slave-tm1g krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390593, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:33.590941 16859 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:36:33.592358 16859 webserver.cc:460] Webserver started at http://127.29.233.131:33031/ using document root <none> and password file <none>
I20260810 19:36:33.593076 16859 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:33.593235 16859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:33.598802 16859 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:36:33.601509 16876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:33.602950 16859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:36:33.603132 16859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "2c3ba879ea9d4d89aaf7d0b0f626d0f9"
format_stamp: "Formatted at 2026-08-10 19:35:47 on dist-test-slave-tm1g"
server_key: "d3efd9235f833bfe9a3e839d12995270"
server_key_iv: "4eb3eea796173991080ebb518791da4b"
server_key_version: "encryptionkey@0"
I20260810 19:36:33.604238 16859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:33.629580 16859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:33.634354 16859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:33.634660 16859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:33.635751 16859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:33.637192 16859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:33.637327 16859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:33.637467 16859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:33.637559 16859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:33.673619 16859 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:37729
I20260810 19:36:33.673724 16997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:37729 every 8 connection(s)
I20260810 19:36:33.675303 16859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:36:33.680567 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:33.678665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:47747)
0810 19:36:33.679441 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:33.679694 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":307,"threads_started":1}
W20260810 19:36:33.681298 16998 heartbeater.cc:644] Failed to heartbeat to 127.29.233.190:42273 (0 consecutive failures): Network error: Failed to ping master at 127.29.233.190:42273: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
I20260810 19:36:33.682193 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:33.681869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:44653)
0810 19:36:33.682020 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:33.682090 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 19:36:33.683143 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:33.682834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:51387)
0810 19:36:33.682957 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:33.683039 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 19:36:33.683422 16998 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 19:36:33.685662 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 16859
I20260810 19:36:33.730988 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:33.730113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:47625)
0810 19:36:33.730633 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:33.730745 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
I20260810 19:36:34.206677 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:34.205773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:43909)
0810 19:36:34.206277 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:34.206431 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":250,"threads_started":1}
I20260810 19:36:34.684772 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:34.683964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:45139)
0810 19:36:34.684452 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:34.684543 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":247,"threads_started":1}
I20260810 19:36:34.732687 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:34.731943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:60207)
0810 19:36:34.732425 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:34.732512 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":249,"threads_started":1}
I20260810 19:36:35.208554 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:35.207776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:47981)
0810 19:36:35.208285 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:35.208373 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":260,"threads_started":1}
I20260810 19:36:35.686728 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:35.685861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:45277)
0810 19:36:35.686470 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:35.686557 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":280,"threads_started":1}
I20260810 19:36:35.734293 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:35.733589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:43667)
0810 19:36:35.734032 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:35.734115 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260810 19:36:36.210233 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:36.209489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:36797)
0810 19:36:36.209968 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:36.210052 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":252,"threads_started":1}
I20260810 19:36:36.688453 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:36.687698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:42275)
0810 19:36:36.688187 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:36.688268 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":261,"threads_started":1}
I20260810 19:36:36.736012 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:36.735255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:42377)
0810 19:36:36.735710 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:36.735795 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":236,"threads_started":1}
I20260810 19:36:37.212123 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:37.211288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:55751)
0810 19:36:37.211800 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:37.211891 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":268,"threads_started":1}
I20260810 19:36:37.690627 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:37.689594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:41881)
0810 19:36:37.690363 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:37.690448 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":467,"threads_started":1}
I20260810 19:36:37.737808 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:37.737046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:59591)
0810 19:36:37.737540 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:37.737626 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":248,"threads_started":1}
I20260810 19:36:38.214047 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:38.213257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:38481)
0810 19:36:38.213771 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:38.213863 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":272,"threads_started":1}
I20260810 19:36:38.692720 17017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:38.691731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:47405)
0810 19:36:38.692324 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:38.692423 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":328,"threads_started":1}
I20260810 19:36:38.739888 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:38.738881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:43055)
0810 19:36:38.739421 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:38.739636 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":293,"threads_started":1}
I20260810 19:36:39.215782 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:39.215007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:60423)
0810 19:36:39.215512 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:39.215605 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":259,"threads_started":1}
I20260810 19:36:39.694662 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:39.693784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:39271)
0810 19:36:39.694362 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:39.694490 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":277,"threads_started":1}
I20260810 19:36:39.741958 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:39.741008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:48695)
0810 19:36:39.741607 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:39.741687 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":291,"threads_started":1}
I20260810 19:36:40.217646 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:40.216806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:58145)
0810 19:36:40.217328 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:40.217415 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":233,"threads_started":1}
I20260810 19:36:40.696731 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:40.695835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:33569)
0810 19:36:40.696413 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:40.696515 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":291,"threads_started":1}
I20260810 19:36:40.743752 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:40.742955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:52253)
0810 19:36:40.743462 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:40.743563 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":271,"threads_started":1}
I20260810 19:36:41.219703 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:41.218891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:49483)
0810 19:36:41.219430 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:41.219526 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":268,"threads_started":1}
I20260810 19:36:41.698494 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:41.697685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:55649)
0810 19:36:41.698155 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:41.698247 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":242,"threads_started":1}
I20260810 19:36:41.746542 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:41.744986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:59931)
0810 19:36:41.746169 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:41.746280 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1031,"thread_start_us":817,"threads_started":1}
I20260810 19:36:42.221518 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:42.220761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:40461)
0810 19:36:42.221245 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:42.221329 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":237,"threads_started":1}
I20260810 19:36:42.700192 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:42.699426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:49731)
0810 19:36:42.699931 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:42.700014 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":247,"threads_started":1}
I20260810 19:36:42.748436 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:42.747631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:51679)
0810 19:36:42.748124 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:42.748217 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":298,"threads_started":1}
I20260810 19:36:43.223555 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:43.222841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:53689)
0810 19:36:43.223287 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:43.223366 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":241,"threads_started":1}
I20260810 19:36:43.702011 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:43.701203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:40407)
0810 19:36:43.701763 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:43.701846 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":305,"threads_started":1}
I20260810 19:36:43.750559 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:43.749502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:58775)
0810 19:36:43.750069 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:43.750212 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":280,"threads_started":1}
I20260810 19:36:44.225565 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:44.224650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:52283)
0810 19:36:44.225202 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:44.225354 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":277,"threads_started":1}
I20260810 19:36:44.703888 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:44.703086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:43589)
0810 19:36:44.703592 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:44.703700 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
I20260810 19:36:44.752318 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:44.751611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:50963)
0810 19:36:44.752054 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:44.752142 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":230,"threads_started":1}
I20260810 19:36:45.227497 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:45.226740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:47647)
0810 19:36:45.227222 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:45.227314 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":237,"threads_started":1}
I20260810 19:36:45.705631 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:45.704890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:59651)
0810 19:36:45.705382 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:45.705468 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":260,"threads_started":1}
I20260810 19:36:45.754184 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:45.753385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:40905)
0810 19:36:45.753879 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:45.753987 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":242,"threads_started":1}
I20260810 19:36:46.229417 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:46.228560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:51715)
0810 19:36:46.229055 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:46.229178 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":226,"threads_started":1}
I20260810 19:36:46.707674 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:46.706791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:34505)
0810 19:36:46.707343 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:46.707474 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":253,"threads_started":1}
I20260810 19:36:46.756031 17045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:46.755235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:44993)
0810 19:36:46.755728 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:46.755825 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":271,"threads_started":1}
I20260810 19:36:47.231590 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:47.230717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:34527)
0810 19:36:47.231303 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:47.231396 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":312,"threads_started":1}
I20260810 19:36:47.709555 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:47.708755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:60689)
0810 19:36:47.709262 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:47.709362 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":259,"threads_started":1}
I20260810 19:36:47.758046 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:47.757271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.129:51339)
0810 19:36:47.757732 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:47.757832 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260810 19:36:48.233760 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:48.232862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.130:59209)
0810 19:36:48.233461 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:48.233575 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":336,"threads_started":1}
I20260810 19:36:48.696689 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 16535
I20260810 19:36:48.711848 17052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:48.710904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:42273 (local address 127.29.233.131:43627)
0810 19:36:48.711506 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:48.711634 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.190:42273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":294,"threads_started":1}
I20260810 19:36:48.716946 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:48.698202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.29.233.129:39027)
0810 19:36:48.698934 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:48.698957 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:48.699012 (+ 55us) server_negotiation.cc:407] Connection header received
0810 19:36:48.700769 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:48.700780 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:48.701222 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:48.701591 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:48.703229 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:48.704499 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:48.705851 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:48.706607 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:48.706810 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:48.709786 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:48.709795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:48.709811 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:48.709864 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:48.713241 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:48.714126 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:48.714134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:48.714142 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:48.714432 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:48.714985 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:48.714989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:48.714992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:48.715618 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:48.715855 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:48.716268 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 19:36:48.716659 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":322,"threads_started":1}
I20260810 19:36:48.718612 16708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:48.810752 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 16535
I20260810 19:36:48.831670 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 16716
I20260810 19:36:48.851532 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:48.832800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.29.233.130:40643)
0810 19:36:48.833396 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:48.833412 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:48.833451 (+ 39us) server_negotiation.cc:407] Connection header received
0810 19:36:48.835558 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:48.835568 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:48.836045 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:48.836350 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:48.837818 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:48.839366 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:48.840749 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:48.841495 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:48.841647 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:48.844679 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:48.844687 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:48.844704 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:48.844749 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:48.847978 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:48.848843 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:48.848851 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:48.848858 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:48.849047 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:48.849520 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:48.849523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:48.849525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:48.850043 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:48.850280 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:48.850776 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 19:36:48.851218 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":289,"threads_started":1}
I20260810 19:36:48.852942 16850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:48.938648 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 16716
I20260810 19:36:48.961242 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 16859
I20260810 19:36:48.980152 17058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:48.962485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.29.233.131:37729)
0810 19:36:48.963115 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:48.963130 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:48.963174 (+ 44us) server_negotiation.cc:407] Connection header received
0810 19:36:48.964814 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:48.964824 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:48.965302 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:48.965670 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:48.967468 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:48.968744 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:48.970013 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:48.970789 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:48.971002 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:48.973787 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:48.973795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:48.973810 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:48.973849 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:48.976848 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:48.977516 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:48.977524 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:48.977530 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:48.977703 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:48.978109 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:48.978113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:48.978115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:48.978771 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:48.979012 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:48.979384 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 19:36:48.979836 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":291,"threads_started":1}
I20260810 19:36:48.981575 16992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:49.066517 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 16859
2026-08-10T19:36:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65160 ms)
[----------] 30 tests from SecurityITest (303906 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:36:49 dist-test-slave-tm1g krb5kdc[17062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:36:49 dist-test-slave-tm1g krb5kdc[17062](info): set up 2 sockets
Aug 10 19:36:49 dist-test-slave-tm1g krb5kdc[17062](info): commencing operation
krb5kdc: starting...
W20260810 19:36:50.272936 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:36:50 dist-test-slave-tm1g krb5kdc[17062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:36:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:36:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:36:50.610631 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35501
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:35501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:50.960278 17079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:50.960604 17079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:50.960789 17079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:50.968250 17079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:36:50.968379 17079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:50.968487 17079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:50.968591 17079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:36:50.968675 17079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:36:50.978758 17079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:35501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:50.980181 17079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:50.981621 17079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:50.994489 17084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:50.994593 17087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:50.994614 17085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:50.995594 17079 server_base.cc:1034] running on GCE node
I20260810 19:36:50.996516 17079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:50.998029 17079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:50.999262 17079 hybrid_clock.cc:648] HybridClock initialized: now 1786390610999188 us; error 62 us; skew 500 ppm
I20260810 19:36:50.999594 17079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:50 dist-test-slave-tm1g krb5kdc[17062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390610, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:51.002857 17079 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:36:51.004328 17079 webserver.cc:460] Webserver started at http://127.29.233.190:40127/ using document root <none> and password file <none>
I20260810 19:36:51.005054 17079 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:51.005194 17079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:51.005548 17079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:51.008633 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "46c1bdb680ba4c95aba27416de4b1de7"
format_stamp: "Formatted at 2026-08-10 19:36:51 on dist-test-slave-tm1g"
server_key: "090ee3e1e58cb9c8184ed58e3db78fe6"
server_key_iv: "30ac6c2c8b16fd5b1875f10b5677c8c7"
server_key_version: "encryptionkey@0"
I20260810 19:36:51.009483 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "46c1bdb680ba4c95aba27416de4b1de7"
format_stamp: "Formatted at 2026-08-10 19:36:51 on dist-test-slave-tm1g"
server_key: "090ee3e1e58cb9c8184ed58e3db78fe6"
server_key_iv: "30ac6c2c8b16fd5b1875f10b5677c8c7"
server_key_version: "encryptionkey@0"
I20260810 19:36:51.014771 17079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 19:36:51.018206 17095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:51.019635 17079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:36:51.019824 17079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "46c1bdb680ba4c95aba27416de4b1de7"
format_stamp: "Formatted at 2026-08-10 19:36:51 on dist-test-slave-tm1g"
server_key: "090ee3e1e58cb9c8184ed58e3db78fe6"
server_key_iv: "30ac6c2c8b16fd5b1875f10b5677c8c7"
server_key_version: "encryptionkey@0"
I20260810 19:36:51.020027 17079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:51.032531 17079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:51.036473 17079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:51.036760 17079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:51.055752 17079 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:35501
I20260810 19:36:51.055763 17151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:35501 every 8 connection(s)
I20260810 19:36:51.057160 17079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:36:51.060874 17154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:51.063148 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 17079
I20260810 19:36:51.063388 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:36:51.064257 30630 external_mini_cluster.cc:1428] Setting key 2324c9cbcfa693e23264ffa4179da5cc
I20260810 19:36:51.073005 17154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7: Bootstrap starting.
I20260810 19:36:51.076261 17154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:51.077286 17154 log.cc:824] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7: Log is configured to *not* fsync() on all Append() calls
Aug 10 19:36:51 dist-test-slave-tm1g krb5kdc[17062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:51.080559 17154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7: No bootstrap required, opened a new log
I20260810 19:36:51.088896 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:51.066565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.29.233.190:35501)
0810 19:36:51.067363 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:51.067378 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:51.067429 (+ 51us) server_negotiation.cc:407] Connection header received
0810 19:36:51.069086 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:51.069107 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:51.069509 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:51.069907 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:51.071459 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:51.072761 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:51.074218 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:51.074988 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:51.075184 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:51.079832 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:51.079842 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:51.079861 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:51.079920 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:51.084095 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:51.084848 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:51.084856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:51.084870 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:51.085108 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:51.085500 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:51.085506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:51.085509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:51.086220 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:51.086573 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:51.086998 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 19:36:51.087466 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":348,"threads_started":1}
I20260810 19:36:51.091518 17154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c1bdb680ba4c95aba27416de4b1de7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35501 } }
I20260810 19:36:51.091990 17154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:51.092113 17154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c1bdb680ba4c95aba27416de4b1de7, State: Initialized, Role: FOLLOWER
I20260810 19:36:51.092657 17154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c1bdb680ba4c95aba27416de4b1de7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35501 } }
I20260810 19:36:51.092865 17154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:36:51.092957 17154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:36:51.093083 17154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:51.095343 17154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c1bdb680ba4c95aba27416de4b1de7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35501 } }
I20260810 19:36:51.095819 17154 leader_election.cc:302] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46c1bdb680ba4c95aba27416de4b1de7; no voters:
I20260810 19:36:51.096401 17154 leader_election.cc:288] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:36:51.096556 17159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:36:51.097950 17159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [term 1 LEADER]: Becoming Leader. State: Replica: 46c1bdb680ba4c95aba27416de4b1de7, State: Running, Role: LEADER
I20260810 19:36:51.098603 17154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:36:51.098506 17159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c1bdb680ba4c95aba27416de4b1de7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35501 } }
I20260810 19:36:51.101367 17162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46c1bdb680ba4c95aba27416de4b1de7. Latest consensus state: current_term: 1 leader_uuid: "46c1bdb680ba4c95aba27416de4b1de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c1bdb680ba4c95aba27416de4b1de7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35501 } } }
I20260810 19:36:51.101800 17162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:51.101414 17160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46c1bdb680ba4c95aba27416de4b1de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c1bdb680ba4c95aba27416de4b1de7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35501 } } }
I20260810 19:36:51.102957 17160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7 [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:51.103803 17168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:36:51.109364 17168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:36:51.118469 17168 catalog_manager.cc:1347] Generated new cluster ID: 24564485633f4b5c976db5662c8bf8a3
I20260810 19:36:51.118613 17168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:36:51.131157 17168 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:36:51.132611 17168 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:36:51.141852 17168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7: Generated new TSK 0
I20260810 19:36:51.142759 17168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:36:51.231153 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:35501
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:51.594014 17181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:51.594357 17181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:51.594544 17181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:51.601991 17181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:51.602124 17181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:51.602345 17181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:36:51.612722 17181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:51.614086 17181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:51.615494 17181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:51.629086 17186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:51.629153 17187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:51.629118 17189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:51.629987 17181 server_base.cc:1034] running on GCE node
I20260810 19:36:51.630678 17181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:51.631649 17181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:51.632884 17181 hybrid_clock.cc:648] HybridClock initialized: now 1786390611632841 us; error 38 us; skew 500 ppm
I20260810 19:36:51.633198 17181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:51 dist-test-slave-tm1g krb5kdc[17062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390611, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:51.636430 17181 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:36:51.637944 17181 webserver.cc:460] Webserver started at http://127.29.233.129:35589/ using document root <none> and password file <none>
I20260810 19:36:51.638721 17181 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:51.638870 17181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:51.639194 17181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:51.642436 17181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "b6802933c8a140e3a2db57b226abfc2d"
format_stamp: "Formatted at 2026-08-10 19:36:51 on dist-test-slave-tm1g"
server_key: "835e8949c970d5a2b825e9eec714ecf5"
server_key_iv: "f7750738acfcfc5e77cc015114da9691"
server_key_version: "encryptionkey@0"
I20260810 19:36:51.643203 17181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "b6802933c8a140e3a2db57b226abfc2d"
format_stamp: "Formatted at 2026-08-10 19:36:51 on dist-test-slave-tm1g"
server_key: "835e8949c970d5a2b825e9eec714ecf5"
server_key_iv: "f7750738acfcfc5e77cc015114da9691"
server_key_version: "encryptionkey@0"
I20260810 19:36:51.648339 17181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260810 19:36:51.651786 17196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:51.653085 17181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:36:51.653267 17181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "b6802933c8a140e3a2db57b226abfc2d"
format_stamp: "Formatted at 2026-08-10 19:36:51 on dist-test-slave-tm1g"
server_key: "835e8949c970d5a2b825e9eec714ecf5"
server_key_iv: "f7750738acfcfc5e77cc015114da9691"
server_key_version: "encryptionkey@0"
I20260810 19:36:51.653471 17181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:51.680514 17181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:51.685050 17181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:51.685385 17181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:51.686507 17181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:51.687932 17181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:51.688032 17181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:51.688155 17181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:51.688218 17181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:51.723587 17181 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:40559
I20260810 19:36:51.723631 17320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:40559 every 8 connection(s)
I20260810 19:36:51.724999 17181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:36:51.729226 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 17181
I20260810 19:36:51.729463 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:36:51.730136 30630 external_mini_cluster.cc:1428] Setting key a974a363e35aff88920fc3c4ed3ec6df
Aug 10 19:36:51 dist-test-slave-tm1g krb5kdc[17062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390611, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:51.748346 17323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:51.727619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:33903 (local address 127.29.233.190:35501)
0810 19:36:51.728128 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:51.728135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:51.729035 (+ 900us) server_negotiation.cc:407] Connection header received
0810 19:36:51.731188 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:51.731193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:51.731293 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:51.731416 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:51.733091 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:51.733824 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:51.735417 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:51.735833 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:51.735982 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:51.740053 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:51.740061 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:51.740069 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:51.740118 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:51.743995 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:51.745161 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:51.745167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:51.745171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:51.745355 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:51.745934 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:51.745943 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:51.745949 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:51.746400 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:51.746557 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:51.747602 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0810 19:36:51.747867 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":236,"threads_started":1}
I20260810 19:36:51.749418 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:51.728167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:35501 (local address 127.29.233.129:33903)
0810 19:36:51.728875 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:51.728915 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:51.730899 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:51.731676 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:51.731684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:51.732357 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:51.732924 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:51.732940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:51.733973 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:51.733978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:51.735273 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:51.735285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:51.736062 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:51.736072 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:51.736313 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:51.737014 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:36:51.737036 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:51.739743 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:51.744200 (+ 4457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:51.744214 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:51.744236 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:51.744998 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:51.745490 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:51.745498 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:51.745504 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:51.745790 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:51.746641 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:51.746653 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:51.747146 (+ 493us) client_negotiation.cc:769] Sending connection context
0810 19:36:51.747432 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 19:36:51.747919 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":274,"threads_started":1}
I20260810 19:36:51.751289 17321 heartbeater.cc:347] Connected to a master server at 127.29.233.190:35501
I20260810 19:36:51.751641 17321 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:51.752588 17321 heartbeater.cc:511] Master 127.29.233.190:35501 requested a full tablet report, sending...
I20260810 19:36:51.754840 17114 ts_manager.cc:194] Registered new tserver with Master: b6802933c8a140e3a2db57b226abfc2d (127.29.233.129:40559)
I20260810 19:36:51.756325 17114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:33903
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:36:51.806749 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:35501
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:52.156558 17327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:52.156899 17327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:52.157075 17327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:52.164306 17327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:52.164472 17327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:52.164660 17327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:36:52.174588 17327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:52.175992 17327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:52.177410 17327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:52.191865 17333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:52.192322 17332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:52.192636 17335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:52.193838 17327 server_base.cc:1034] running on GCE node
I20260810 19:36:52.194586 17327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:52.195597 17327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:52.196858 17327 hybrid_clock.cc:648] HybridClock initialized: now 1786390612196780 us; error 42 us; skew 500 ppm
I20260810 19:36:52.197167 17327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:52 dist-test-slave-tm1g krb5kdc[17062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390612, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:52.200734 17327 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:36:52.202214 17327 webserver.cc:460] Webserver started at http://127.29.233.130:42451/ using document root <none> and password file <none>
I20260810 19:36:52.203011 17327 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:52.203116 17327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:52.203383 17327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:52.206715 17327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "26fc73476cad43ff84c027951879ab3e"
format_stamp: "Formatted at 2026-08-10 19:36:52 on dist-test-slave-tm1g"
server_key: "bfaa7805ba0487000752faf6f9aa8647"
server_key_iv: "1906780ed614f3ae1bc820a6321fa389"
server_key_version: "encryptionkey@0"
I20260810 19:36:52.207461 17327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "26fc73476cad43ff84c027951879ab3e"
format_stamp: "Formatted at 2026-08-10 19:36:52 on dist-test-slave-tm1g"
server_key: "bfaa7805ba0487000752faf6f9aa8647"
server_key_iv: "1906780ed614f3ae1bc820a6321fa389"
server_key_version: "encryptionkey@0"
I20260810 19:36:52.212821 17327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 19:36:52.216331 17343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:52.217729 17327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:36:52.217864 17327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "26fc73476cad43ff84c027951879ab3e"
format_stamp: "Formatted at 2026-08-10 19:36:52 on dist-test-slave-tm1g"
server_key: "bfaa7805ba0487000752faf6f9aa8647"
server_key_iv: "1906780ed614f3ae1bc820a6321fa389"
server_key_version: "encryptionkey@0"
I20260810 19:36:52.218032 17327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:52.241353 17327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:52.245817 17327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:52.246080 17327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:52.247171 17327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:52.248508 17327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:52.248589 17327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:52.248677 17327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:52.248720 17327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:52.284201 17327 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:33735
I20260810 19:36:52.284312 17467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:33735 every 8 connection(s)
I20260810 19:36:52.285665 17327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:36:52.290232 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 17327
I20260810 19:36:52.290498 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:36:52.291157 30630 external_mini_cluster.cc:1428] Setting key 9580522f902ead2a2d78d0dcd380ac6d
Aug 10 19:36:52 dist-test-slave-tm1g krb5kdc[17062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390612, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:52.309283 17470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.288397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:37177 (local address 127.29.233.190:35501)
0810 19:36:52.288887 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:52.288892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:52.289906 (+ 1014us) server_negotiation.cc:407] Connection header received
0810 19:36:52.292382 (+ 2476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:52.292388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:52.292493 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:52.292628 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:52.294619 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.295368 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.297924 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.298462 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.298633 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.303010 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:52.303023 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:52.303027 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:52.303063 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:52.306114 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:52.306920 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:52.306925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:52.306929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:52.307072 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:52.307514 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:52.307519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:52.307522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:52.307799 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:52.307958 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:52.308665 (+ 707us) server_negotiation.cc:299] Negotiation successful
0810 19:36:52.308898 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":250,"threads_started":1}
I20260810 19:36:52.310463 17471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.288968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:35501 (local address 127.29.233.130:37177)
0810 19:36:52.289699 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:52.289736 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:52.292093 (+ 2357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:52.292870 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:52.292879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:52.293538 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:52.294417 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.294445 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.295526 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.295534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.297762 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.297779 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.298742 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.298751 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.299096 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.299938 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:36:52.299975 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:52.302756 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:52.306283 (+ 3527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:52.306292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:52.306336 (+ 44us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:52.306775 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:52.307189 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:52.307194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:52.307197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:52.307394 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:52.307931 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:52.307939 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:52.308287 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 19:36:52.308567 (+ 280us) client_negotiation.cc:240] Negotiation successful
0810 19:36:52.309129 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":280,"threads_started":1}
I20260810 19:36:52.312222 17468 heartbeater.cc:347] Connected to a master server at 127.29.233.190:35501
I20260810 19:36:52.312602 17468 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:52.313577 17468 heartbeater.cc:511] Master 127.29.233.190:35501 requested a full tablet report, sending...
I20260810 19:36:52.315673 17114 ts_manager.cc:194] Registered new tserver with Master: 26fc73476cad43ff84c027951879ab3e (127.29.233.130:33735)
I20260810 19:36:52.316702 17114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:37177
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:36:52.366633 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:35501
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:52.713745 17474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:52.714035 17474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:52.714197 17474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:52.721896 17474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:52.722012 17474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:52.722179 17474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:36:52.732702 17474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:35501
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:52.734004 17474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:52.735461 17474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:52.749011 17480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:52.749150 17479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:52.749178 17482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:52.750635 17474 server_base.cc:1034] running on GCE node
I20260810 19:36:52.751247 17474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:52.752311 17474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:52.753552 17474 hybrid_clock.cc:648] HybridClock initialized: now 1786390612753508 us; error 35 us; skew 500 ppm
I20260810 19:36:52.753881 17474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:36:52.759579 17321 heartbeater.cc:503] Master 127.29.233.190:35501 was elected leader, sending a full tablet report...
Aug 10 19:36:52 dist-test-slave-tm1g krb5kdc[17062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390612, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:52.761021 17474 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:36:52.762802 17474 webserver.cc:460] Webserver started at http://127.29.233.131:45445/ using document root <none> and password file <none>
I20260810 19:36:52.763492 17474 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:52.763600 17474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:52.763852 17474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:52.766888 17474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "5ea816cd448d46ae8aca45dfb309fae2"
format_stamp: "Formatted at 2026-08-10 19:36:52 on dist-test-slave-tm1g"
server_key: "a51fbc0934e5a9bdc140108c9480b07d"
server_key_iv: "ce2b600dfe92c61d3a36478856043641"
server_key_version: "encryptionkey@0"
I20260810 19:36:52.767608 17474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "5ea816cd448d46ae8aca45dfb309fae2"
format_stamp: "Formatted at 2026-08-10 19:36:52 on dist-test-slave-tm1g"
server_key: "a51fbc0934e5a9bdc140108c9480b07d"
server_key_iv: "ce2b600dfe92c61d3a36478856043641"
server_key_version: "encryptionkey@0"
I20260810 19:36:52.772977 17474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:36:52.776539 17489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:52.778002 17474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:36:52.778187 17474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "5ea816cd448d46ae8aca45dfb309fae2"
format_stamp: "Formatted at 2026-08-10 19:36:52 on dist-test-slave-tm1g"
server_key: "a51fbc0934e5a9bdc140108c9480b07d"
server_key_iv: "ce2b600dfe92c61d3a36478856043641"
server_key_version: "encryptionkey@0"
I20260810 19:36:52.778475 17474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:52.794793 17474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:52.798961 17474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:52.799309 17474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:52.800348 17474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:52.801749 17474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:52.801862 17474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:52.802014 17474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:52.802098 17474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:52.839558 17474 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:36387
I20260810 19:36:52.839594 17611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:36387 every 8 connection(s)
I20260810 19:36:52.840930 17474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:36:52.849764 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 17474
I20260810 19:36:52.849982 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:36:52.850629 30630 external_mini_cluster.cc:1428] Setting key 8f3596231ecf8397eb6a3aa6beaa9a57
Aug 10 19:36:52 dist-test-slave-tm1g krb5kdc[17062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390612, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:52.863205 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.844325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:33959 (local address 127.29.233.190:35501)
0810 19:36:52.844819 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:52.844825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:52.845822 (+ 997us) server_negotiation.cc:407] Connection header received
0810 19:36:52.847765 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:52.847769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:52.847847 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:52.847958 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:52.849436 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.850731 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.852872 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.853299 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.853446 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.857063 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:52.857068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:52.857072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:52.857107 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:52.859981 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:52.860741 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:52.860751 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:52.860753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:52.860878 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:52.861316 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:52.861340 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:52.861342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:52.861629 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:52.861802 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:52.862568 (+ 766us) server_negotiation.cc:299] Negotiation successful
0810 19:36:52.862806 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":245,"threads_started":1}
I20260810 19:36:52.864174 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.844919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:35501 (local address 127.29.233.131:33959)
0810 19:36:52.845649 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:52.845699 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:52.847506 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:52.848163 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:52.848170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:52.848693 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:52.849257 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.849273 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.850889 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.850892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.852706 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.852723 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.853541 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.853550 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.853829 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.854546 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:36:52.854568 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:52.856846 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:52.860135 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:52.860144 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:52.860162 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:52.860603 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:52.860992 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:52.860997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:52.861001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:52.861198 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:52.861762 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:52.861769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:52.862129 (+ 360us) client_negotiation.cc:769] Sending connection context
0810 19:36:52.862475 (+ 346us) client_negotiation.cc:240] Negotiation successful
0810 19:36:52.863034 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":287,"threads_started":1}
I20260810 19:36:52.866063 17612 heartbeater.cc:347] Connected to a master server at 127.29.233.190:35501
I20260810 19:36:52.866472 17612 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:52.867347 17612 heartbeater.cc:511] Master 127.29.233.190:35501 requested a full tablet report, sending...
I20260810 19:36:52.868942 17113 ts_manager.cc:194] Registered new tserver with Master: 5ea816cd448d46ae8aca45dfb309fae2 (127.29.233.131:36387)
I20260810 19:36:52.869903 17113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:33959
I20260810 19:36:52.879632 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:36:52.898407 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.884868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.29.233.190:35501)
0810 19:36:52.885100 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:52.885108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:52.885573 (+ 465us) server_negotiation.cc:407] Connection header received
0810 19:36:52.885775 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:52.885781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:52.885864 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:52.885985 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:52.887635 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.888387 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.889794 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.890215 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.890375 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.892834 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:52.892841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:52.892846 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:52.892887 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:52.895570 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:52.896262 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:52.896269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:52.896273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:52.896406 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:52.896813 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:52.896821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:52.896825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:52.897158 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:52.897407 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:52.897752 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 19:36:52.897987 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 19:36:52.902827 17113 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:36:52.909960 17113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:36:52.934898 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.922918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33735 (local address 127.0.0.1:43682)
0810 19:36:52.924809 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:52.924829 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:52.925022 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:52.926452 (+ 1430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:52.926553 (+ 101us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:52.926607 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:52.927111 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.927129 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.929389 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.929398 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.931787 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.931802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.933804 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.933810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.934014 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.934083 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 19:36:52.934208 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 19:36:52.934448 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1751,"thread_start_us":690,"threads_started":1}
I20260810 19:36:52.935745 17628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.923600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.29.233.130:33735)
0810 19:36:52.925345 (+ 1745us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:52.925355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:52.925481 (+ 126us) server_negotiation.cc:407] Connection header received
0810 19:36:52.925607 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:52.925618 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:52.925892 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:52.926119 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:52.927473 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.929237 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.932094 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.933607 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.933780 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.935172 (+ 1392us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:52.935262 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 19:36:52.935444 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1547,"thread_start_us":1445,"threads_started":1}
I20260810 19:36:52.936791 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.922477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59304)
0810 19:36:52.924101 (+ 1624us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:52.924163 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:52.925254 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:52.928231 (+ 2977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:52.928238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:52.928270 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:52.928647 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.928656 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.930650 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.930654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.933375 (+ 2721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.933390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.935771 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.935778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.936045 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.936113 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 19:36:52.936230 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 19:36:52.936375 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1293,"mutex_wait_us":61,"thread_start_us":1956,"threads_started":1}
I20260810 19:36:52.936786 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.922867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.29.233.129:40559)
0810 19:36:52.927336 (+ 4469us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:52.927348 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:52.927378 (+ 30us) server_negotiation.cc:407] Connection header received
0810 19:36:52.927484 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:52.927492 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:52.927746 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:52.927971 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:52.928838 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.930475 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.933556 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.935442 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.935641 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.935803 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:52.936277 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 19:36:52.936456 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4299,"thread_start_us":4125,"threads_started":1}
I20260810 19:36:52.936403 17629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.925158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:36387 (local address 127.0.0.1:50102)
0810 19:36:52.928189 (+ 3031us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:52.928208 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:52.928332 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:52.928947 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:52.928954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:52.928983 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:52.929595 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.929611 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.932066 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.932073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.933649 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:52.933663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.935502 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.935509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:52.935684 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.935730 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 19:36:52.935835 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 19:36:52.935996 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2902,"mutex_wait_us":45,"thread_start_us":2770,"threads_started":1}
I20260810 19:36:52.938171 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:52.925798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.29.233.131:36387)
0810 19:36:52.927756 (+ 1958us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:52.927764 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:52.928318 (+ 554us) server_negotiation.cc:407] Connection header received
0810 19:36:52.928519 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:52.928530 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:52.928742 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:52.928947 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:52.929806 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.931871 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.933834 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:52.935228 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:52.935417 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:52.935605 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:52.936509 (+ 904us) server_negotiation.cc:299] Negotiation successful
0810 19:36:52.936679 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1797,"thread_start_us":1658,"threads_started":1}
I20260810 19:36:52.939843 17396 tablet_service.cc:1467] Processing CreateTablet for tablet 0cc9af4675f6467b85f3843953dca200 (DEFAULT_TABLE table=test-table [id=79fb678792e64bf18d59f56ef7ecf524]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:52.940390 17250 tablet_service.cc:1467] Processing CreateTablet for tablet 0cc9af4675f6467b85f3843953dca200 (DEFAULT_TABLE table=test-table [id=79fb678792e64bf18d59f56ef7ecf524]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:52.941227 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cc9af4675f6467b85f3843953dca200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:52.941840 17250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cc9af4675f6467b85f3843953dca200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:52.941668 17542 tablet_service.cc:1467] Processing CreateTablet for tablet 0cc9af4675f6467b85f3843953dca200 (DEFAULT_TABLE table=test-table [id=79fb678792e64bf18d59f56ef7ecf524]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:52.943102 17542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cc9af4675f6467b85f3843953dca200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:52.954288 17631 tablet_bootstrap.cc:492] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Bootstrap starting.
I20260810 19:36:52.956411 17632 tablet_bootstrap.cc:492] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: Bootstrap starting.
I20260810 19:36:52.958209 17633 tablet_bootstrap.cc:492] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Bootstrap starting.
I20260810 19:36:52.958731 17631 tablet_bootstrap.cc:654] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:52.960032 17632 tablet_bootstrap.cc:654] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:52.960135 17631 log.cc:824] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:52.961151 17633 tablet_bootstrap.cc:654] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:52.961365 17632 log.cc:824] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:52.962369 17633 log.cc:824] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:52.963932 17631 tablet_bootstrap.cc:492] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: No bootstrap required, opened a new log
I20260810 19:36:52.964226 17631 ts_tablet_manager.cc:1397] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 19:36:52.965092 17633 tablet_bootstrap.cc:492] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: No bootstrap required, opened a new log
I20260810 19:36:52.965174 17632 tablet_bootstrap.cc:492] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: No bootstrap required, opened a new log
I20260810 19:36:52.965378 17633 ts_tablet_manager.cc:1397] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 19:36:52.965461 17632 ts_tablet_manager.cc:1397] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260810 19:36:52.972442 17633 raft_consensus.cc:348] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:52.972903 17633 raft_consensus.cc:374] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:52.973022 17633 raft_consensus.cc:729] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea816cd448d46ae8aca45dfb309fae2, State: Initialized, Role: FOLLOWER
I20260810 19:36:52.973740 17633 consensus_queue.cc:260] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:52.974922 17612 heartbeater.cc:503] Master 127.29.233.190:35501 was elected leader, sending a full tablet report...
I20260810 19:36:52.975028 17631 raft_consensus.cc:348] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:52.975445 17631 raft_consensus.cc:374] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:52.975531 17631 raft_consensus.cc:729] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26fc73476cad43ff84c027951879ab3e, State: Initialized, Role: FOLLOWER
I20260810 19:36:52.975595 17633 ts_tablet_manager.cc:1428] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260810 19:36:52.976085 17632 raft_consensus.cc:348] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:52.976207 17631 consensus_queue.cc:260] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:52.976495 17632 raft_consensus.cc:374] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:52.976608 17632 raft_consensus.cc:729] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6802933c8a140e3a2db57b226abfc2d, State: Initialized, Role: FOLLOWER
I20260810 19:36:52.977356 17468 heartbeater.cc:503] Master 127.29.233.190:35501 was elected leader, sending a full tablet report...
I20260810 19:36:52.977334 17632 consensus_queue.cc:260] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:52.978178 17631 ts_tablet_manager.cc:1428] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 19:36:52.979810 17632 ts_tablet_manager.cc:1428] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
W20260810 19:36:53.037799 17469 tablet.cc:2369] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:36:53.080247 17637 raft_consensus.cc:482] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:36:53.080533 17637 raft_consensus.cc:504] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:53.082414 17637 leader_election.cc:288] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6802933c8a140e3a2db57b226abfc2d (127.29.233.129:40559), 26fc73476cad43ff84c027951879ab3e (127.29.233.130:33735)
I20260810 19:36:53.089473 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.082719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.29.233.131:33141)
0810 19:36:53.082941 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.082962 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:53.083113 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:53.083851 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:53.083856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:53.083883 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:53.084122 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:53.084131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.085342 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.085345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:53.087358 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:53.087373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.088751 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.088755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:53.088940 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.088965 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 19:36:53.089048 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 19:36:53.089141 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":78}
I20260810 19:36:53.089509 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.083189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:33141 (local address 127.29.233.129:40559)
0810 19:36:53.083558 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:53.083563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:53.083582 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:36:53.083645 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:53.083649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:53.083707 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:53.083833 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:53.084286 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.085198 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.087533 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.088608 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.088771 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.088852 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:53.089107 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 19:36:53.089260 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260810 19:36:53.090127 17628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.083089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:60817 (local address 127.29.233.130:33735)
0810 19:36:53.083305 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:53.083311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:53.083855 (+ 544us) server_negotiation.cc:407] Connection header received
0810 19:36:53.084159 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:53.084163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:53.084221 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:53.084319 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:53.084862 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.085740 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.087971 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.089148 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.089290 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.089354 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:53.089762 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 19:36:53.089878 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:36:53.090611 17272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc9af4675f6467b85f3843953dca200" candidate_uuid: "5ea816cd448d46ae8aca45dfb309fae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6802933c8a140e3a2db57b226abfc2d" is_pre_election: true
I20260810 19:36:53.090914 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.082834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:33735 (local address 127.29.233.131:60817)
0810 19:36:53.083743 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.083761 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:53.084000 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:53.084333 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:53.084337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:53.084359 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:53.084678 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:53.084686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.085890 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.085894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:53.087809 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:53.087820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.089359 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.089364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:53.089558 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.089598 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 19:36:53.090339 (+ 741us) client_negotiation.cc:240] Negotiation successful
0810 19:36:53.090506 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":1,"spinlock_wait_cycles":24960,"thread_start_us":578,"threads_started":1}
I20260810 19:36:53.091277 17272 raft_consensus.cc:2454] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea816cd448d46ae8aca45dfb309fae2 in term 0.
I20260810 19:36:53.092039 17493 leader_election.cc:302] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ea816cd448d46ae8aca45dfb309fae2, b6802933c8a140e3a2db57b226abfc2d; no voters:
I20260810 19:36:53.091950 17418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc9af4675f6467b85f3843953dca200" candidate_uuid: "5ea816cd448d46ae8aca45dfb309fae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26fc73476cad43ff84c027951879ab3e" is_pre_election: true
I20260810 19:36:53.092473 17637 raft_consensus.cc:2781] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:36:53.092492 17418 raft_consensus.cc:2454] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea816cd448d46ae8aca45dfb309fae2 in term 0.
I20260810 19:36:53.092638 17637 raft_consensus.cc:482] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:36:53.092772 17637 raft_consensus.cc:3037] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 0 FOLLOWER]: Advancing to term 1
W20260810 19:36:53.092906 17613 tablet.cc:2369] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:36:53.095409 17637 raft_consensus.cc:504] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:53.096009 17637 leader_election.cc:288] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [CANDIDATE]: Term 1 election: Requested vote from peers b6802933c8a140e3a2db57b226abfc2d (127.29.233.129:40559), 26fc73476cad43ff84c027951879ab3e (127.29.233.130:33735)
I20260810 19:36:53.096486 17272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc9af4675f6467b85f3843953dca200" candidate_uuid: "5ea816cd448d46ae8aca45dfb309fae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6802933c8a140e3a2db57b226abfc2d"
I20260810 19:36:53.096537 17418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc9af4675f6467b85f3843953dca200" candidate_uuid: "5ea816cd448d46ae8aca45dfb309fae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26fc73476cad43ff84c027951879ab3e"
I20260810 19:36:53.096695 17272 raft_consensus.cc:3037] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:53.096760 17418 raft_consensus.cc:3037] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:53.099177 17418 raft_consensus.cc:2454] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea816cd448d46ae8aca45dfb309fae2 in term 1.
I20260810 19:36:53.099186 17272 raft_consensus.cc:2454] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea816cd448d46ae8aca45dfb309fae2 in term 1.
I20260810 19:36:53.099635 17491 leader_election.cc:302] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26fc73476cad43ff84c027951879ab3e, 5ea816cd448d46ae8aca45dfb309fae2; no voters:
I20260810 19:36:53.099961 17637 raft_consensus.cc:2781] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:36:53.100405 17637 raft_consensus.cc:686] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 1 LEADER]: Becoming Leader. State: Replica: 5ea816cd448d46ae8aca45dfb309fae2, State: Running, Role: LEADER
I20260810 19:36:53.100894 17637 consensus_queue.cc:237] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } }
I20260810 19:36:53.105985 17114 catalog_manager.cc:5495] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ea816cd448d46ae8aca45dfb309fae2 (127.29.233.131). New cstate: current_term: 1 leader_uuid: "5ea816cd448d46ae8aca45dfb309fae2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ea816cd448d46ae8aca45dfb309fae2" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 36387 } health_report { overall_health: HEALTHY } } }
I20260810 19:36:53.125594 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.120004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.29.233.131:36387)
0810 19:36:53.120203 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:53.120208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:53.120227 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:36:53.120361 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:53.120365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:53.120461 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:53.120593 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:36:53.121003 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.121846 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.123435 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.123890 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.124017 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.124521 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:36:53.125084 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:36:53.125199 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:53.125319 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 19:36:53.125387 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:36:53.134380 17418 raft_consensus.cc:1261] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 1 FOLLOWER]: Refusing update from remote peer 5ea816cd448d46ae8aca45dfb309fae2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:36:53.134366 17272 raft_consensus.cc:1261] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 1 FOLLOWER]: Refusing update from remote peer 5ea816cd448d46ae8aca45dfb309fae2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:36:53.135538 17637 consensus_queue.cc:1035] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26fc73476cad43ff84c027951879ab3e" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 33735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:36:53.135864 17641 consensus_queue.cc:1035] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6802933c8a140e3a2db57b226abfc2d" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:36:53.148407 17646 mvcc.cc:205] Tried to move back new op lower bound from 7317055951387148288 to 7317055951265021952. Current Snapshot: MvccSnapshot[applied={T|T < 7317055951387148288}]
I20260810 19:36:53.153126 17645 mvcc.cc:205] Tried to move back new op lower bound from 7317055951387148288 to 7317055951265021952. Current Snapshot: MvccSnapshot[applied={T|T < 7317055951387148288}]
I20260810 19:36:53.160209 17113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:36:53.160499 17113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:36:53.164126 17113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46c1bdb680ba4c95aba27416de4b1de7: Sending DeleteTablet for 3 replicas of tablet 0cc9af4675f6467b85f3843953dca200
I20260810 19:36:53.165419 17542 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cc9af4675f6467b85f3843953dca200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:53 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:50102
I20260810 19:36:53.165452 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cc9af4675f6467b85f3843953dca200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:53 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:43682
I20260810 19:36:53.165583 17250 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cc9af4675f6467b85f3843953dca200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:53 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:59304
I20260810 19:36:53.167435 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 17181
I20260810 19:36:53.177898 17655 tablet_replica.cc:330] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: stopping tablet replica
I20260810 19:36:53.178632 17655 raft_consensus.cc:2229] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:36:53.179358 17655 raft_consensus.cc:2258] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:53.182159 17655 ts_tablet_manager.cc:1905] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:53.182942 17656 tablet_replica.cc:330] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: stopping tablet replica
I20260810 19:36:53.183521 17656 raft_consensus.cc:2229] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:36:53.191359 17655 ts_tablet_manager.cc:1918] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:53.192215 17656 raft_consensus.cc:2258] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:53.194911 17656 ts_tablet_manager.cc:1905] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 19:36:53 dist-test-slave-tm1g krb5kdc[17062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:36:53.197418 17655 log.cc:1196] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/0cc9af4675f6467b85f3843953dca200
I20260810 19:36:53.197608 17657 tablet_replica.cc:330] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: stopping tablet replica
I20260810 19:36:53.198107 17655 ts_tablet_manager.cc:1939] T 0cc9af4675f6467b85f3843953dca200 P 26fc73476cad43ff84c027951879ab3e: Deleting consensus metadata
I20260810 19:36:53.198084 17657 raft_consensus.cc:2229] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:36:53.198652 17657 raft_consensus.cc:2258] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:53.200245 17096 catalog_manager.cc:4841] TS 26fc73476cad43ff84c027951879ab3e (127.29.233.130:33735): tablet 0cc9af4675f6467b85f3843953dca200 (table test-table [id=79fb678792e64bf18d59f56ef7ecf524]) successfully deleted
I20260810 19:36:53.202598 17657 ts_tablet_manager.cc:1905] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:53.203244 17656 ts_tablet_manager.cc:1918] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:53.203498 17656 log.cc:1196] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/0cc9af4675f6467b85f3843953dca200
I20260810 19:36:53.204447 17656 ts_tablet_manager.cc:1939] T 0cc9af4675f6467b85f3843953dca200 P 5ea816cd448d46ae8aca45dfb309fae2: Deleting consensus metadata
I20260810 19:36:53.206440 17099 catalog_manager.cc:4841] TS 5ea816cd448d46ae8aca45dfb309fae2 (127.29.233.131:36387): tablet 0cc9af4675f6467b85f3843953dca200 (table test-table [id=79fb678792e64bf18d59f56ef7ecf524]) successfully deleted
I20260810 19:36:53.209215 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.169298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.29.233.129:40559)
0810 19:36:53.169532 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:53.169541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:53.185384 (+ 15843us) server_negotiation.cc:407] Connection header received
0810 19:36:53.185885 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:53.185891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:53.185975 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:53.186050 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:53.187913 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.189186 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.191313 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.192007 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.192115 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.200869 (+ 8754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:53.200887 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:53.200906 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:53.200979 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:53.205373 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:53.206476 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:53.206486 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:53.206490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:53.206676 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:53.207237 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:53.207243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:53.207246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:53.207770 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:53.208110 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:53.208515 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 19:36:53.208795 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:36:53.209942 17316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:53.301522 17657 ts_tablet_manager.cc:1918] T 0cc9af4675f6467b85f3843953dca200 P b6802933c8a140e3a2db57b226abfc2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:53.301633 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 17181
I20260810 19:36:53.327775 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 17327
W20260810 19:36:53.327747 17099 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.233.129:40559 (error 108)
I20260810 19:36:53.330116 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.329594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59318)
0810 19:36:53.329864 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.329967 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260810 19:36:53.330587 17099 catalog_manager.cc:4584] TS b6802933c8a140e3a2db57b226abfc2d (127.29.233.129:40559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0cc9af4675f6467b85f3843953dca200: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Aug 10 19:36:53 dist-test-slave-tm1g krb5kdc[17062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:36:53.346882 17628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.329148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.29.233.130:33735)
0810 19:36:53.329368 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:53.329374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:53.329392 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:36:53.329463 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:53.329468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:53.329551 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:53.329674 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:53.331516 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.332297 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.334291 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.334865 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.334968 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.340130 (+ 5162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:53.340146 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:53.340162 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:53.340198 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:53.343751 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:53.344663 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:53.344670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:53.344672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:53.344797 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:53.345223 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:53.345226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:53.345228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:53.345638 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:53.345876 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:53.346247 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 19:36:53.346592 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:36:53.347616 17463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:53.384550 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.383999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59326)
0810 19:36:53.384238 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.384350 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 19:36:53.385743 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.385326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59330)
0810 19:36:53.385500 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.385604 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 19:36:53.446889 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.446340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59334)
0810 19:36:53.446575 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.446691 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 19:36:53.447377 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 17327
I20260810 19:36:53.451059 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.450109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59336)
0810 19:36:53.450872 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.450946 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":660}
I20260810 19:36:53.468516 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 17474
Aug 10 19:36:53 dist-test-slave-tm1g krb5kdc[17062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:36:53.485625 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.469379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50108 (local address 127.29.233.131:36387)
0810 19:36:53.469580 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:53.469588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:53.469610 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:36:53.469824 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:53.469830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:53.469918 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:53.470041 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:53.471496 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.472318 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.473603 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:53.474089 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:53.474239 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:53.479002 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:53.479014 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:53.479037 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:53.479076 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:53.482590 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:53.483308 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:53.483319 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:53.483331 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:53.483534 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:53.483949 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:53.483955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:53.483959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:53.484350 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:53.484682 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:53.484974 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 19:36:53.485212 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 19:36:53.486363 17607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:53.531895 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.531371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59344)
0810 19:36:53.531605 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.531704 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 19:36:53.533008 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.532635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59360)
0810 19:36:53.532834 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.532902 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260810 19:36:53.614166 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 17474
I20260810 19:36:53.634474 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 17079
I20260810 19:36:53.635730 17146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:53.751073 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:53.750416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40559 (local address 127.0.0.1:59368)
0810 19:36:53.750645 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:53.750778 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 19:36:53.751225 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 17079
2026-08-10T19:36:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4689 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T19:36:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:36:53Z Disabled control of system clock
I20260810 19:36:53.852881 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:40581
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:40581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 19:36:54.198697 17668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:54.198995 17668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:54.199079 17668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:54.206450 17668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:36:54.206564 17668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:54.206602 17668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:54.206661 17668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:36:54.206705 17668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:36:54.215957 17668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:40581
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:40581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:54.217319 17668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:54.219128 17668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:54.231519 17673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:54.231506 17674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:54.232084 17676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:54.232745 17668 server_base.cc:1034] running on GCE node
I20260810 19:36:54.233595 17668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:54.235092 17668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:54.236294 17668 hybrid_clock.cc:648] HybridClock initialized: now 1786390614236234 us; error 52 us; skew 500 ppm
I20260810 19:36:54.236608 17668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:36:54.238967 17668 webserver.cc:460] Webserver started at http://127.29.233.190:33023/ using document root <none> and password file <none>
I20260810 19:36:54.239656 17668 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:54.239748 17668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:54.240015 17668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:54.242997 17668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "35b177e38b274f93a404cdc9394607ac"
format_stamp: "Formatted at 2026-08-10 19:36:54 on dist-test-slave-tm1g"
server_key: "b2a9e680429a2bca11c50dfe428c4084"
server_key_iv: "6c6cf1013e49163eb3e225a7039bb639"
server_key_version: "encryptionkey@0"
I20260810 19:36:54.243829 17668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "35b177e38b274f93a404cdc9394607ac"
format_stamp: "Formatted at 2026-08-10 19:36:54 on dist-test-slave-tm1g"
server_key: "b2a9e680429a2bca11c50dfe428c4084"
server_key_iv: "6c6cf1013e49163eb3e225a7039bb639"
server_key_version: "encryptionkey@0"
I20260810 19:36:54.248999 17668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:36:54.252483 17682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:54.253791 17668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:36:54.253991 17668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "35b177e38b274f93a404cdc9394607ac"
format_stamp: "Formatted at 2026-08-10 19:36:54 on dist-test-slave-tm1g"
server_key: "b2a9e680429a2bca11c50dfe428c4084"
server_key_iv: "6c6cf1013e49163eb3e225a7039bb639"
server_key_version: "encryptionkey@0"
I20260810 19:36:54.254218 17668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:54.266179 17668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:54.267014 17668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:54.267311 17668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:54.286599 17668 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:40581
I20260810 19:36:54.286607 17737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:40581 every 8 connection(s)
I20260810 19:36:54.287959 17668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:36:54.291230 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 17668
I20260810 19:36:54.291426 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:36:54.292049 30630 external_mini_cluster.cc:1428] Setting key 9883ccaa68b001e03bef27d468a66aae
I20260810 19:36:54.292342 17738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:54.307013 17738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac: Bootstrap starting.
I20260810 19:36:54.307015 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:54.293969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50408 (local address 127.29.233.190:40581)
0810 19:36:54.295007 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:54.295017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:54.295062 (+ 45us) server_negotiation.cc:407] Connection header received
0810 19:36:54.297032 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:54.297053 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:54.297487 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:54.297867 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:54.298583 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:54.300007 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:54.301867 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:54.302944 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:54.303185 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:54.304529 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:54.304537 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:54.304550 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:54.305203 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:54.305283 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:54.305315 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:54.305407 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:54.305632 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 19:36:54.305984 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":435,"threads_started":1}
I20260810 19:36:54.310140 17738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:54.311161 17738 log.cc:824] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:54.313860 17738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac: No bootstrap required, opened a new log
I20260810 19:36:54.321557 17738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b177e38b274f93a404cdc9394607ac" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40581 } }
I20260810 19:36:54.321980 17738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:54.322078 17738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b177e38b274f93a404cdc9394607ac, State: Initialized, Role: FOLLOWER
I20260810 19:36:54.322741 17738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b177e38b274f93a404cdc9394607ac" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40581 } }
I20260810 19:36:54.322984 17738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:36:54.323150 17738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:36:54.323354 17738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:54.325625 17738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b177e38b274f93a404cdc9394607ac" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40581 } }
I20260810 19:36:54.326241 17738 leader_election.cc:302] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35b177e38b274f93a404cdc9394607ac; no voters:
I20260810 19:36:54.326997 17738 leader_election.cc:288] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:36:54.327118 17746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:36:54.327888 17746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [term 1 LEADER]: Becoming Leader. State: Replica: 35b177e38b274f93a404cdc9394607ac, State: Running, Role: LEADER
I20260810 19:36:54.328429 17746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b177e38b274f93a404cdc9394607ac" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40581 } }
I20260810 19:36:54.329141 17738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:36:54.331615 17748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35b177e38b274f93a404cdc9394607ac. Latest consensus state: current_term: 1 leader_uuid: "35b177e38b274f93a404cdc9394607ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b177e38b274f93a404cdc9394607ac" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40581 } } }
I20260810 19:36:54.331566 17747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35b177e38b274f93a404cdc9394607ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35b177e38b274f93a404cdc9394607ac" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40581 } } }
I20260810 19:36:54.332019 17748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:54.332096 17747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:54.339263 17755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:36:54.344368 17755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:36:54.353952 17755 catalog_manager.cc:1347] Generated new cluster ID: d700877975ab45fb9b08458bc2495770
I20260810 19:36:54.354084 17755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:36:54.367998 17755 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:36:54.369249 17755 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:36:54.379210 17755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac: Generated new TSK 0
I20260810 19:36:54.380045 17755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:36:54.388131 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:40581
--builtin_ntp_servers=127.29.233.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 19:36:54.737685 17768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:54.737960 17768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:54.738056 17768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:54.745477 17768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:54.745575 17768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:54.745679 17768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:36:54.755595 17768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:40581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:54.756887 17768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:54.758937 17768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:54.772554 17774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:54.772516 17777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:54.772590 17775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:54.773550 17768 server_base.cc:1034] running on GCE node
I20260810 19:36:54.774174 17768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:54.775170 17768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:54.776420 17768 hybrid_clock.cc:648] HybridClock initialized: now 1786390614776367 us; error 64 us; skew 500 ppm
I20260810 19:36:54.776743 17768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:36:54.779017 17768 webserver.cc:460] Webserver started at http://127.29.233.129:34937/ using document root <none> and password file <none>
I20260810 19:36:54.779740 17768 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:54.779878 17768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:54.780205 17768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:54.783164 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "7b831ee1c51b4b34a50329880c96c498"
format_stamp: "Formatted at 2026-08-10 19:36:54 on dist-test-slave-tm1g"
server_key: "c780704ba7cd7485a5964187077372e3"
server_key_iv: "526ed0cc4f18b57d0038de43625dc800"
server_key_version: "encryptionkey@0"
I20260810 19:36:54.784057 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "7b831ee1c51b4b34a50329880c96c498"
format_stamp: "Formatted at 2026-08-10 19:36:54 on dist-test-slave-tm1g"
server_key: "c780704ba7cd7485a5964187077372e3"
server_key_iv: "526ed0cc4f18b57d0038de43625dc800"
server_key_version: "encryptionkey@0"
I20260810 19:36:54.789273 17768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:36:54.792769 17785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:54.794154 17768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:36:54.794394 17768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "7b831ee1c51b4b34a50329880c96c498"
format_stamp: "Formatted at 2026-08-10 19:36:54 on dist-test-slave-tm1g"
server_key: "c780704ba7cd7485a5964187077372e3"
server_key_iv: "526ed0cc4f18b57d0038de43625dc800"
server_key_version: "encryptionkey@0"
I20260810 19:36:54.794625 17768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:54.821017 17768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:54.821942 17768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:54.822275 17768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:54.823371 17768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:54.825017 17768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:54.825150 17768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:54.825279 17768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:54.825389 17768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:54.860001 17768 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:42299
I20260810 19:36:54.860092 17908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:42299 every 8 connection(s)
I20260810 19:36:54.861352 17768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:36:54.865449 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 17768
I20260810 19:36:54.865639 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:36:54.866156 30630 external_mini_cluster.cc:1428] Setting key edaa5a618de75eaf8fbc6bad2d5958c9
I20260810 19:36:54.869333 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:40581
--builtin_ntp_servers=127.29.233.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 19:36:54.875377 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:54.864023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:51699 (local address 127.29.233.190:40581)
0810 19:36:54.864491 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:54.864495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:54.865841 (+ 1346us) server_negotiation.cc:407] Connection header received
0810 19:36:54.868223 (+ 2382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:54.868227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:54.868313 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:54.868431 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:54.869470 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:54.870656 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:54.872188 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:54.872913 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:54.873055 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:54.874393 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:54.874401 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:54.874409 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:54.874434 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:54.874498 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:54.874517 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:54.874613 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:54.874826 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 19:36:54.874947 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260810 19:36:54.876170 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:54.864634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:40581 (local address 127.29.233.129:51699)
0810 19:36:54.865628 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:54.865699 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:54.867883 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:54.868632 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:54.868641 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:54.868722 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:54.869277 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:54.869293 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:54.870809 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:54.870813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:54.872036 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:54.872049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:54.873132 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:54.873137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:54.873338 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:54.874057 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:36:54.874060 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:54.874103 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:36:54.874175 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:54.874623 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:54.874633 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:54.874639 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 19:36:54.874789 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 19:36:54.875082 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":297,"threads_started":1}
I20260810 19:36:54.877821 17909 heartbeater.cc:347] Connected to a master server at 127.29.233.190:40581
I20260810 19:36:54.878149 17909 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:54.879105 17909 heartbeater.cc:511] Master 127.29.233.190:40581 requested a full tablet report, sending...
I20260810 19:36:54.881348 17699 ts_manager.cc:194] Registered new tserver with Master: 7b831ee1c51b4b34a50329880c96c498 (127.29.233.129:42299)
I20260810 19:36:54.883051 17699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.129:51699
W20260810 19:36:55.216310 17913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:55.216621 17913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:55.216710 17913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:55.223935 17913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:55.224042 17913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:55.224149 17913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:36:55.234053 17913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:40581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:55.235520 17913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:55.237462 17913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:55.251282 17918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:55.251287 17922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:55.251318 17919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:55.252200 17913 server_base.cc:1034] running on GCE node
I20260810 19:36:55.252782 17913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:55.253684 17913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:55.254925 17913 hybrid_clock.cc:648] HybridClock initialized: now 1786390615254875 us; error 44 us; skew 500 ppm
I20260810 19:36:55.255304 17913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:36:55.257637 17913 webserver.cc:460] Webserver started at http://127.29.233.130:40309/ using document root <none> and password file <none>
I20260810 19:36:55.258512 17913 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:55.258643 17913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:55.258996 17913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:55.261940 17913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "0c5be75df24b4f65b0283496b6cd741b"
format_stamp: "Formatted at 2026-08-10 19:36:55 on dist-test-slave-tm1g"
server_key: "b0c00878fdfdec30294ca7e8e9e5ddc0"
server_key_iv: "e70311db44dc7fc7a8b0e576eaeddf24"
server_key_version: "encryptionkey@0"
I20260810 19:36:55.262766 17913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "0c5be75df24b4f65b0283496b6cd741b"
format_stamp: "Formatted at 2026-08-10 19:36:55 on dist-test-slave-tm1g"
server_key: "b0c00878fdfdec30294ca7e8e9e5ddc0"
server_key_iv: "e70311db44dc7fc7a8b0e576eaeddf24"
server_key_version: "encryptionkey@0"
I20260810 19:36:55.268187 17913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:36:55.271791 17930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:55.273090 17913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:36:55.273270 17913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "0c5be75df24b4f65b0283496b6cd741b"
format_stamp: "Formatted at 2026-08-10 19:36:55 on dist-test-slave-tm1g"
server_key: "b0c00878fdfdec30294ca7e8e9e5ddc0"
server_key_iv: "e70311db44dc7fc7a8b0e576eaeddf24"
server_key_version: "encryptionkey@0"
I20260810 19:36:55.273490 17913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:55.291678 17913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:55.292658 17913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:55.292975 17913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:55.294064 17913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:55.295619 17913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:55.295742 17913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:55.295876 17913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:55.295964 17913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:55.331142 17913 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:38567
I20260810 19:36:55.331198 18047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:38567 every 8 connection(s)
I20260810 19:36:55.332509 17913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:36:55.333155 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 17913
I20260810 19:36:55.333351 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:36:55.333994 30630 external_mini_cluster.cc:1428] Setting key 9aea2252d7d7c61a03668dc2c3cff7ea
I20260810 19:36:55.337541 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:40581
--builtin_ntp_servers=127.29.233.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 19:36:55.347822 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.335389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:44903 (local address 127.29.233.190:40581)
0810 19:36:55.335598 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:55.335605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:55.337329 (+ 1724us) server_negotiation.cc:407] Connection header received
0810 19:36:55.340205 (+ 2876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:55.340209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:55.340278 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:55.340358 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:55.341745 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.342553 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.344980 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.345363 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.345455 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.347091 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:55.347095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:55.347099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:55.347117 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:55.347149 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:55.347161 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:55.347231 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:55.347506 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 19:36:55.347589 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:36:55.349261 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.335937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:40581 (local address 127.29.233.130:44903)
0810 19:36:55.337159 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:55.337218 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:55.339900 (+ 2682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:55.340618 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:55.340632 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:55.340744 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:55.341571 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.341599 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.342699 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.342707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.344828 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.344848 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.345588 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.345598 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.345939 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.346730 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:36:55.346735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:55.346787 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:36:55.346892 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:55.347260 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:55.347272 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:55.347282 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 19:36:55.347471 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 19:36:55.347886 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":406,"threads_started":1}
I20260810 19:36:55.350708 18048 heartbeater.cc:347] Connected to a master server at 127.29.233.190:40581
I20260810 19:36:55.351020 18048 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:55.351814 18048 heartbeater.cc:511] Master 127.29.233.190:40581 requested a full tablet report, sending...
I20260810 19:36:55.353325 17699 ts_manager.cc:194] Registered new tserver with Master: 0c5be75df24b4f65b0283496b6cd741b (127.29.233.130:38567)
I20260810 19:36:55.354615 17699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.130:44903
W20260810 19:36:55.683394 18052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:55.683696 18052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:55.683784 18052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:55.690948 18052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:55.691049 18052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:55.691157 18052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:36:55.700301 18052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:40581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:55.701617 18052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:55.703392 18052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:55.716672 18057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:55.716694 18060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:55.716777 18058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:55.718070 18052 server_base.cc:1034] running on GCE node
I20260810 19:36:55.718789 18052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:55.719784 18052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:55.720994 18052 hybrid_clock.cc:648] HybridClock initialized: now 1786390615720944 us; error 45 us; skew 500 ppm
I20260810 19:36:55.721313 18052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:36:55.723639 18052 webserver.cc:460] Webserver started at http://127.29.233.131:38641/ using document root <none> and password file <none>
I20260810 19:36:55.724377 18052 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:55.724498 18052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:55.724838 18052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:55.727806 18052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "418322d131f64d0baa3b8b18674d5924"
format_stamp: "Formatted at 2026-08-10 19:36:55 on dist-test-slave-tm1g"
server_key: "596d6f77cf9ce09d57c7d7f42ad7a6db"
server_key_iv: "12b0e82dcff08d11477ba2c677fdde03"
server_key_version: "encryptionkey@0"
I20260810 19:36:55.728624 18052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "418322d131f64d0baa3b8b18674d5924"
format_stamp: "Formatted at 2026-08-10 19:36:55 on dist-test-slave-tm1g"
server_key: "596d6f77cf9ce09d57c7d7f42ad7a6db"
server_key_iv: "12b0e82dcff08d11477ba2c677fdde03"
server_key_version: "encryptionkey@0"
I20260810 19:36:55.733827 18052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:36:55.737346 18067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:55.738855 18052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:36:55.739046 18052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "418322d131f64d0baa3b8b18674d5924"
format_stamp: "Formatted at 2026-08-10 19:36:55 on dist-test-slave-tm1g"
server_key: "596d6f77cf9ce09d57c7d7f42ad7a6db"
server_key_iv: "12b0e82dcff08d11477ba2c677fdde03"
server_key_version: "encryptionkey@0"
I20260810 19:36:55.739272 18052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:55.760078 18052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:55.760911 18052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:55.761293 18052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:55.762355 18052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:55.763859 18052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:55.764000 18052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:55.764137 18052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:55.764233 18052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:55.798763 18052 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:34547
I20260810 19:36:55.798816 18187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:34547 every 8 connection(s)
I20260810 19:36:55.800138 18052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:36:55.802134 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 18052
I20260810 19:36:55.802361 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:36:55.802840 30630 external_mini_cluster.cc:1428] Setting key 7347455de5b6cab77dedfdde00fd8cf1
I20260810 19:36:55.812676 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.803857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:59609 (local address 127.29.233.190:40581)
0810 19:36:55.804047 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:55.804053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:55.804529 (+ 476us) server_negotiation.cc:407] Connection header received
0810 19:36:55.806573 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:55.806577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:55.806646 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:55.806720 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:55.807691 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.808590 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.810101 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.810578 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.810676 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.811952 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:55.811958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:55.811961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:55.811981 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:55.812018 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:55.812031 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:55.812120 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:55.812313 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 19:36:55.812410 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 19:36:55.813630 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.803254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:40581 (local address 127.29.233.131:59609)
0810 19:36:55.804352 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:55.804404 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:55.806257 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:55.806906 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:55.806913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:55.806995 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:55.807552 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.807568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.808722 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.808726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.809971 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.809983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.810768 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.810773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.810971 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.811613 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:36:55.811616 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:55.811652 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:36:55.811734 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:55.812127 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:55.812134 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:55.812140 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 19:36:55.812276 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 19:36:55.812583 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":359,"threads_started":1}
I20260810 19:36:55.815053 18188 heartbeater.cc:347] Connected to a master server at 127.29.233.190:40581
I20260810 19:36:55.815352 18188 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:55.816126 18188 heartbeater.cc:511] Master 127.29.233.190:40581 requested a full tablet report, sending...
I20260810 19:36:55.817637 17699 ts_manager.cc:194] Registered new tserver with Master: 418322d131f64d0baa3b8b18674d5924 (127.29.233.131:34547)
I20260810 19:36:55.818450 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:36:55.818687 17699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.131:59609
I20260810 19:36:55.829221 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.823418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.29.233.190:40581)
0810 19:36:55.823583 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:55.823587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:55.824155 (+ 568us) server_negotiation.cc:407] Connection header received
0810 19:36:55.824261 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:55.824264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:55.824339 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:55.824419 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:55.824881 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.825577 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.826954 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.827360 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.827454 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.828422 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:55.828427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:55.828430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:55.828447 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:55.828479 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:55.828491 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:55.828570 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:55.828786 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 19:36:55.828872 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 19:36:55.832906 17699 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:36:55.839951 17699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:36:55.875988 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.852250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.29.233.130:38567)
0810 19:36:55.864621 (+ 12371us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:55.864630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:55.864662 (+ 32us) server_negotiation.cc:407] Connection header received
0810 19:36:55.864765 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:55.864774 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:55.865035 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:55.865222 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:55.866185 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.868839 (+ 2654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.873593 (+ 4754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.874726 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.874914 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.875143 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:55.875439 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 19:36:55.875622 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12216,"thread_start_us":12126,"threads_started":1}
I20260810 19:36:55.877238 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.852113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:38567 (local address 127.0.0.1:45748)
0810 19:36:55.862943 (+ 10830us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:55.862990 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:55.863210 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:55.865525 (+ 2315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:55.865533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:55.865564 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:55.865964 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.865982 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.869012 (+ 3030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.869018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.873352 (+ 4334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.873371 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.874923 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.874929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.875160 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.875229 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 19:36:55.875391 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 19:36:55.876840 (+ 1449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10522,"mutex_wait_us":1225,"thread_start_us":19996,"threads_started":1}
I20260810 19:36:55.880148 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.852310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:34547 (local address 127.0.0.1:37148)
0810 19:36:55.871542 (+ 19232us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:55.871563 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:55.871680 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:55.872687 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:55.872694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:55.872719 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:55.873092 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.873104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.875052 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.875061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.877452 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.877468 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.879318 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.879327 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.879530 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.879562 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 19:36:55.879662 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 19:36:55.879785 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19072,"mutex_wait_us":138,"thread_start_us":18937,"threads_started":1}
I20260810 19:36:55.880272 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.853095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.29.233.131:34547)
0810 19:36:55.871084 (+ 17989us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:55.871093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:55.871820 (+ 727us) server_negotiation.cc:407] Connection header received
0810 19:36:55.871927 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:55.871934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:55.872200 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:55.872391 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:55.873326 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.874890 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.877694 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.879086 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.879310 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.879527 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:55.879723 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 19:36:55.879891 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17802,"thread_start_us":17617,"threads_started":1}
I20260810 19:36:55.880920 18202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.854074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:42299 (local address 127.0.0.1:55606)
0810 19:36:55.872414 (+ 18340us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:55.872436 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:55.872570 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:55.873222 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:55.873227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:55.873259 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:55.873722 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.873735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.875569 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.875575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.877901 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:55.877916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.880153 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.880160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:55.880357 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.880397 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 19:36:55.880490 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 19:36:55.880628 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18190,"mutex_wait_us":24,"thread_start_us":8176,"threads_started":1}
I20260810 19:36:55.881714 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:55.854411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.29.233.129:42299)
0810 19:36:55.870556 (+ 16145us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:55.870572 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:55.872563 (+ 1991us) server_negotiation.cc:407] Connection header received
0810 19:36:55.872763 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:55.872774 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:55.873030 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:55.873228 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:55.873903 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.875419 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.878121 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:55.879568 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:55.879808 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:55.880005 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:55.881207 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0810 19:36:55.881348 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15972,"thread_start_us":15865,"threads_started":1}
I20260810 19:36:55.881531 17980 tablet_service.cc:1467] Processing CreateTablet for tablet 108c712d06664fb4a0429706979190b0 (DEFAULT_TABLE table=test-table [id=259a158fd0114807b53068c37fff2c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:55.883147 17980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108c712d06664fb4a0429706979190b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:55.884258 18120 tablet_service.cc:1467] Processing CreateTablet for tablet 108c712d06664fb4a0429706979190b0 (DEFAULT_TABLE table=test-table [id=259a158fd0114807b53068c37fff2c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:55.885794 18120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108c712d06664fb4a0429706979190b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:55.885355 17838 tablet_service.cc:1467] Processing CreateTablet for tablet 108c712d06664fb4a0429706979190b0 (DEFAULT_TABLE table=test-table [id=259a158fd0114807b53068c37fff2c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:36:55.886600 17909 heartbeater.cc:503] Master 127.29.233.190:40581 was elected leader, sending a full tablet report...
I20260810 19:36:55.886767 17838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 108c712d06664fb4a0429706979190b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:55.895249 18209 tablet_bootstrap.cc:492] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Bootstrap starting.
I20260810 19:36:55.898973 18209 tablet_bootstrap.cc:654] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:55.900355 18209 log.cc:824] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:55.900507 18210 tablet_bootstrap.cc:492] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Bootstrap starting.
I20260810 19:36:55.902259 18211 tablet_bootstrap.cc:492] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Bootstrap starting.
I20260810 19:36:55.904470 18210 tablet_bootstrap.cc:654] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:55.905383 18211 tablet_bootstrap.cc:654] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:55.905761 18210 log.cc:824] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:55.905987 18209 tablet_bootstrap.cc:492] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: No bootstrap required, opened a new log
I20260810 19:36:55.906273 18209 ts_tablet_manager.cc:1397] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260810 19:36:55.906597 18211 log.cc:824] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:55.908881 18210 tablet_bootstrap.cc:492] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: No bootstrap required, opened a new log
I20260810 19:36:55.909152 18210 ts_tablet_manager.cc:1397] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 19:36:55.909690 18211 tablet_bootstrap.cc:492] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: No bootstrap required, opened a new log
I20260810 19:36:55.909950 18211 ts_tablet_manager.cc:1397] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 19:36:55.915138 18209 raft_consensus.cc:348] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:55.915531 18209 raft_consensus.cc:374] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:55.915644 18209 raft_consensus.cc:729] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c5be75df24b4f65b0283496b6cd741b, State: Initialized, Role: FOLLOWER
I20260810 19:36:55.916260 18209 consensus_queue.cc:260] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:55.917397 18048 heartbeater.cc:503] Master 127.29.233.190:40581 was elected leader, sending a full tablet report...
I20260810 19:36:55.918121 18209 ts_tablet_manager.cc:1428] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Time spent starting tablet: real 0.012s user 0.005s sys 0.005s
I20260810 19:36:55.919971 18210 raft_consensus.cc:348] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:55.920452 18210 raft_consensus.cc:374] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:55.920569 18210 raft_consensus.cc:729] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418322d131f64d0baa3b8b18674d5924, State: Initialized, Role: FOLLOWER
I20260810 19:36:55.920514 18211 raft_consensus.cc:348] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:55.920914 18211 raft_consensus.cc:374] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:55.921002 18211 raft_consensus.cc:729] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b831ee1c51b4b34a50329880c96c498, State: Initialized, Role: FOLLOWER
I20260810 19:36:55.921188 18210 consensus_queue.cc:260] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:55.921569 18211 consensus_queue.cc:260] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:55.922999 18188 heartbeater.cc:503] Master 127.29.233.190:40581 was elected leader, sending a full tablet report...
I20260810 19:36:55.923918 18210 ts_tablet_manager.cc:1428] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260810 19:36:55.926931 18211 ts_tablet_manager.cc:1428] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
W20260810 19:36:56.051921 18189 tablet.cc:2369] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:36:56.084863 18049 tablet.cc:2369] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:36:56.114204 17910 tablet.cc:2369] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:36:56.150614 18219 raft_consensus.cc:482] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:36:56.150904 18219 raft_consensus.cc:504] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:56.152599 18219 leader_election.cc:288] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c5be75df24b4f65b0283496b6cd741b (127.29.233.130:38567), 418322d131f64d0baa3b8b18674d5924 (127.29.233.131:34547)
I20260810 19:36:56.160081 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.153111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:33333 (local address 127.29.233.131:34547)
0810 19:36:56.153313 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:56.153321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:56.153996 (+ 675us) server_negotiation.cc:407] Connection header received
0810 19:36:56.154250 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:56.154255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:56.154358 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:56.154487 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:56.155390 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.156288 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.158206 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.159145 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.159331 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.159404 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:56.159696 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 19:36:56.159830 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 19:36:56.160179 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.153209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:59059 (local address 127.29.233.130:38567)
0810 19:36:56.153439 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:56.153449 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:56.154080 (+ 631us) server_negotiation.cc:407] Connection header received
0810 19:36:56.154227 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:56.154233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:56.154357 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:56.154461 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:36:56.155217 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.156147 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.158279 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.159253 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.159427 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.159498 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:56.159796 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 19:36:56.159934 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 19:36:56.160368 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.152938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:34547 (local address 127.29.233.129:33333)
0810 19:36:56.153860 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:56.153889 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:56.154067 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:56.154649 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:56.154653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:56.154689 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:56.155223 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:56.155237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.156440 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.156445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:56.158009 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:56.158021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.159336 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.159342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:56.159528 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.159554 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 19:36:56.159659 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 19:36:56.159786 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":27,"thread_start_us":585,"threads_started":1}
I20260810 19:36:56.160501 18220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.153033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:38567 (local address 127.29.233.129:59059)
0810 19:36:56.153969 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:56.153993 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:56.154089 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:56.154531 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:56.154536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:56.154566 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:36:56.154997 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:56.155007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.156277 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.156284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:56.158023 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:56.158036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.159456 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.159463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:56.159621 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.159644 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:36:56.159746 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 19:36:56.159869 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":2,"thread_start_us":413,"threads_started":1}
I20260810 19:36:56.161674 18141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108c712d06664fb4a0429706979190b0" candidate_uuid: "7b831ee1c51b4b34a50329880c96c498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418322d131f64d0baa3b8b18674d5924" is_pre_election: true
I20260810 19:36:56.161674 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108c712d06664fb4a0429706979190b0" candidate_uuid: "7b831ee1c51b4b34a50329880c96c498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c5be75df24b4f65b0283496b6cd741b" is_pre_election: true
I20260810 19:36:56.162353 18001 raft_consensus.cc:2454] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b831ee1c51b4b34a50329880c96c498 in term 0.
I20260810 19:36:56.162353 18141 raft_consensus.cc:2454] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b831ee1c51b4b34a50329880c96c498 in term 0.
I20260810 19:36:56.163103 17788 leader_election.cc:302] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5be75df24b4f65b0283496b6cd741b, 7b831ee1c51b4b34a50329880c96c498; no voters:
I20260810 19:36:56.163573 18219 raft_consensus.cc:2781] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:36:56.163735 18219 raft_consensus.cc:482] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:36:56.163851 18219 raft_consensus.cc:3037] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:56.166091 18219 raft_consensus.cc:504] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:56.166728 18219 leader_election.cc:288] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [CANDIDATE]: Term 1 election: Requested vote from peers 0c5be75df24b4f65b0283496b6cd741b (127.29.233.130:38567), 418322d131f64d0baa3b8b18674d5924 (127.29.233.131:34547)
I20260810 19:36:56.167208 18141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108c712d06664fb4a0429706979190b0" candidate_uuid: "7b831ee1c51b4b34a50329880c96c498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418322d131f64d0baa3b8b18674d5924"
I20260810 19:36:56.167210 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108c712d06664fb4a0429706979190b0" candidate_uuid: "7b831ee1c51b4b34a50329880c96c498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c5be75df24b4f65b0283496b6cd741b"
I20260810 19:36:56.167397 18141 raft_consensus.cc:3037] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:56.167443 18001 raft_consensus.cc:3037] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:56.169643 18001 raft_consensus.cc:2454] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b831ee1c51b4b34a50329880c96c498 in term 1.
I20260810 19:36:56.169849 18141 raft_consensus.cc:2454] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b831ee1c51b4b34a50329880c96c498 in term 1.
I20260810 19:36:56.170100 17788 leader_election.cc:302] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5be75df24b4f65b0283496b6cd741b, 7b831ee1c51b4b34a50329880c96c498; no voters:
I20260810 19:36:56.170406 18219 raft_consensus.cc:2781] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:36:56.170926 18219 raft_consensus.cc:686] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 1 LEADER]: Becoming Leader. State: Replica: 7b831ee1c51b4b34a50329880c96c498, State: Running, Role: LEADER
I20260810 19:36:56.171403 18219 consensus_queue.cc:237] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } }
I20260810 19:36:56.176203 17698 catalog_manager.cc:5495] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b831ee1c51b4b34a50329880c96c498 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "7b831ee1c51b4b34a50329880c96c498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b831ee1c51b4b34a50329880c96c498" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 42299 } health_report { overall_health: HEALTHY } } }
I20260810 19:36:56.239673 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.233984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55620 (local address 127.29.233.129:42299)
0810 19:36:56.234189 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:56.234196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:56.234279 (+ 83us) server_negotiation.cc:407] Connection header received
0810 19:36:56.234571 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:56.234575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:56.234680 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:56.234842 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:36:56.235247 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.235934 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.237330 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.237770 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.237906 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.238484 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:36:56.239053 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:36:56.239215 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:56.239297 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 19:36:56.239379 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:36:56.248235 18001 raft_consensus.cc:1261] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 1 FOLLOWER]: Refusing update from remote peer 7b831ee1c51b4b34a50329880c96c498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:36:56.248235 18141 raft_consensus.cc:1261] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 1 FOLLOWER]: Refusing update from remote peer 7b831ee1c51b4b34a50329880c96c498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:36:56.249328 18222 consensus_queue.cc:1035] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c5be75df24b4f65b0283496b6cd741b" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 38567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:36:56.249696 18219 consensus_queue.cc:1035] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "418322d131f64d0baa3b8b18674d5924" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:36:56.261471 18226 mvcc.cc:205] Tried to move back new op lower bound from 7317055964142002176 to 7317055963841843200. Current Snapshot: MvccSnapshot[applied={T|T < 7317055964142002176}]
I20260810 19:36:56.265079 18228 mvcc.cc:205] Tried to move back new op lower bound from 7317055964142002176 to 7317055963841843200. Current Snapshot: MvccSnapshot[applied={T|T < 7317055964142002176}]
I20260810 19:36:56.275473 17699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:36:56.275754 17699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:36:56.279348 17699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35b177e38b274f93a404cdc9394607ac: Sending DeleteTablet for 3 replicas of tablet 108c712d06664fb4a0429706979190b0
I20260810 19:36:56.280560 17980 tablet_service.cc:1514] Processing DeleteTablet for tablet 108c712d06664fb4a0429706979190b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:56 UTC) from {username='slave'} at 127.0.0.1:45748
I20260810 19:36:56.280561 18120 tablet_service.cc:1514] Processing DeleteTablet for tablet 108c712d06664fb4a0429706979190b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:56 UTC) from {username='slave'} at 127.0.0.1:37148
I20260810 19:36:56.280654 17838 tablet_service.cc:1514] Processing DeleteTablet for tablet 108c712d06664fb4a0429706979190b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:36:56 UTC) from {username='slave'} at 127.0.0.1:55606
I20260810 19:36:56.284281 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 17768
I20260810 19:36:56.293138 18238 tablet_replica.cc:330] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: stopping tablet replica
I20260810 19:36:56.293890 18239 tablet_replica.cc:330] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: stopping tablet replica
I20260810 19:36:56.294713 18239 raft_consensus.cc:2229] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:36:56.295363 18238 raft_consensus.cc:2229] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:36:56.296098 18238 raft_consensus.cc:2258] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:56.297083 18239 raft_consensus.cc:2258] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:56.300411 18238 ts_tablet_manager.cc:1905] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:56.301415 18237 tablet_replica.cc:330] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: stopping tablet replica
I20260810 19:36:56.301884 18237 raft_consensus.cc:2229] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:36:56.302462 18237 raft_consensus.cc:2258] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:36:56.303030 18239 ts_tablet_manager.cc:1905] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:56.304581 18237 ts_tablet_manager.cc:1905] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:36:56.308331 18238 ts_tablet_manager.cc:1918] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:56.308509 18238 log.cc:1196] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/108c712d06664fb4a0429706979190b0
I20260810 19:36:56.309105 18238 ts_tablet_manager.cc:1939] T 108c712d06664fb4a0429706979190b0 P 0c5be75df24b4f65b0283496b6cd741b: Deleting consensus metadata
I20260810 19:36:56.311352 17685 catalog_manager.cc:4841] TS 0c5be75df24b4f65b0283496b6cd741b (127.29.233.130:38567): tablet 108c712d06664fb4a0429706979190b0 (table test-table [id=259a158fd0114807b53068c37fff2c3e]) successfully deleted
I20260810 19:36:56.311904 18237 ts_tablet_manager.cc:1918] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:56.312125 18237 log.cc:1196] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/108c712d06664fb4a0429706979190b0
I20260810 19:36:56.312737 18237 ts_tablet_manager.cc:1939] T 108c712d06664fb4a0429706979190b0 P 418322d131f64d0baa3b8b18674d5924: Deleting consensus metadata
I20260810 19:36:56.314584 17686 catalog_manager.cc:4841] TS 418322d131f64d0baa3b8b18674d5924 (127.29.233.131:34547): tablet 108c712d06664fb4a0429706979190b0 (table test-table [id=259a158fd0114807b53068c37fff2c3e]) successfully deleted
I20260810 19:36:56.316017 18239 ts_tablet_manager.cc:1918] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:36:56.316215 18239 log.cc:1196] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/108c712d06664fb4a0429706979190b0
I20260810 19:36:56.316582 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.285964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.29.233.129:42299)
0810 19:36:56.286167 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:56.286173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:56.299702 (+ 13529us) server_negotiation.cc:407] Connection header received
0810 19:36:56.299966 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:56.299973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:56.300086 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:56.300166 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:56.306831 (+ 6665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.307954 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.312247 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.312944 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.313089 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.314763 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:56.314774 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:56.314795 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:56.315588 (+ 793us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:56.315644 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:56.315670 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:56.315773 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:56.316031 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 19:36:56.316138 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 19:36:56.316815 18239 ts_tablet_manager.cc:1939] T 108c712d06664fb4a0429706979190b0 P 7b831ee1c51b4b34a50329880c96c498: Deleting consensus metadata
I20260810 19:36:56.317452 17903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:56.414517 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 17768
I20260810 19:36:56.414644 17683 catalog_manager.cc:4841] TS 7b831ee1c51b4b34a50329880c96c498 (127.29.233.129:42299): tablet 108c712d06664fb4a0429706979190b0 (table test-table [id=259a158fd0114807b53068c37fff2c3e]) successfully deleted
I20260810 19:36:56.436342 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 17913
I20260810 19:36:56.443678 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.437175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45762 (local address 127.29.233.130:38567)
0810 19:36:56.437351 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:56.437355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:56.437374 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:36:56.437530 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:56.437534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:56.437623 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:56.437710 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:56.438148 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.439041 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.440470 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.440894 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.440994 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.442060 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:56.442067 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:56.442082 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:56.442861 (+ 779us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:56.442909 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:56.442930 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:56.443018 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:56.443178 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 19:36:56.443279 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 19:36:56.444427 18042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:56.533311 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 17913
I20260810 19:36:56.555438 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 18052
I20260810 19:36:56.562621 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:56.556248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.29.233.131:34547)
0810 19:36:56.556422 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:56.556427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:56.556446 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:36:56.556607 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:56.556610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:56.556711 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:56.556801 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:56.557176 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.557960 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.559448 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:56.559982 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:56.560089 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:56.561154 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:56.561164 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:56.561179 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:36:56.561798 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:56.561852 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:36:56.561873 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:56.561987 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:56.562146 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 19:36:56.562244 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 19:36:56.563498 18183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:56.651187 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 18052
I20260810 19:36:56.670389 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 17668
I20260810 19:36:56.671638 17732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:36:56.783131 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 17668
2026-08-10T19:36:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3032 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:36:56 dist-test-slave-tm1g krb5kdc[18247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:36:56 dist-test-slave-tm1g krb5kdc[18247](info): set up 2 sockets
Aug 10 19:36:56 dist-test-slave-tm1g krb5kdc[18247](info): commencing operation
krb5kdc: starting...
W20260810 19:36:58.023556 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:36:58 dist-test-slave-tm1g krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:36:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:36:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:36:58.358610 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:45773
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:45773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:58.708427 18264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:58.708730 18264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:58.708832 18264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:58.716130 18264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:36:58.716239 18264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:58.716286 18264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:58.716344 18264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:36:58.716389 18264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:36:58.726435 18264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:45773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:45773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:58.727780 18264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:58.729058 18264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:58.741168 18269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:58.741127 18272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:58.741083 18270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:58.742537 18264 server_base.cc:1034] running on GCE node
I20260810 19:36:58.743305 18264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:58.744724 18264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:58.745967 18264 hybrid_clock.cc:648] HybridClock initialized: now 1786390618745920 us; error 35 us; skew 500 ppm
I20260810 19:36:58.746402 18264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:58 dist-test-slave-tm1g krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390618, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:58.753357 18264 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:36:58.755201 18264 webserver.cc:460] Webserver started at http://127.29.233.190:44919/ using document root <none> and password file <none>
I20260810 19:36:58.755981 18264 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:58.756138 18264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:58.756474 18264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:58.760298 18264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "92f6ad691fbd4d4f8f04548bb5428a9e"
format_stamp: "Formatted at 2026-08-10 19:36:58 on dist-test-slave-tm1g"
server_key: "44c11a2c1a48e1fba025bb729f18b671"
server_key_iv: "67bcc7b15d969de3e61752b7a42c7d3a"
server_key_version: "encryptionkey@0"
I20260810 19:36:58.761220 18264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "92f6ad691fbd4d4f8f04548bb5428a9e"
format_stamp: "Formatted at 2026-08-10 19:36:58 on dist-test-slave-tm1g"
server_key: "44c11a2c1a48e1fba025bb729f18b671"
server_key_iv: "67bcc7b15d969de3e61752b7a42c7d3a"
server_key_version: "encryptionkey@0"
I20260810 19:36:58.766788 18264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 19:36:58.770262 18279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:58.771792 18264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:36:58.772001 18264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "92f6ad691fbd4d4f8f04548bb5428a9e"
format_stamp: "Formatted at 2026-08-10 19:36:58 on dist-test-slave-tm1g"
server_key: "44c11a2c1a48e1fba025bb729f18b671"
server_key_iv: "67bcc7b15d969de3e61752b7a42c7d3a"
server_key_version: "encryptionkey@0"
I20260810 19:36:58.772207 18264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:58.793262 18264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:58.797550 18264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:58.797868 18264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:58.817199 18264 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:45773
I20260810 19:36:58.817229 18337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:45773 every 8 connection(s)
I20260810 19:36:58.818615 18264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:36:58.821758 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 18264
I20260810 19:36:58.822016 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:36:58.822515 18338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:36:58.822621 30630 external_mini_cluster.cc:1428] Setting key 6eeb30063062cbd18a0f9158b5329c5b
I20260810 19:36:58.837363 18338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e: Bootstrap starting.
Aug 10 19:36:58 dist-test-slave-tm1g krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:58.841199 18338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e: Neither blocks nor log segments found. Creating new log.
I20260810 19:36:58.842272 18338 log.cc:824] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e: Log is configured to *not* fsync() on all Append() calls
I20260810 19:36:58.845829 18338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e: No bootstrap required, opened a new log
I20260810 19:36:58.848119 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:58.824555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.29.233.190:45773)
0810 19:36:58.825451 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:58.825466 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:58.825522 (+ 56us) server_negotiation.cc:407] Connection header received
0810 19:36:58.827740 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:58.827772 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:58.828334 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:58.828787 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:58.830219 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:58.832273 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:58.833675 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:58.834849 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:58.835064 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:58.839745 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:58.839753 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:58.839767 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:58.839813 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:58.843406 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:58.844292 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:58.844298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:58.844304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:58.844489 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:58.844987 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:58.844991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:58.844993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:58.845694 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:58.846036 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:58.846646 (+ 610us) server_negotiation.cc:299] Negotiation successful
0810 19:36:58.847070 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":355,"threads_started":1}
I20260810 19:36:58.854622 18338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45773 } }
I20260810 19:36:58.854995 18338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:36:58.855115 18338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f6ad691fbd4d4f8f04548bb5428a9e, State: Initialized, Role: FOLLOWER
I20260810 19:36:58.855724 18338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45773 } }
I20260810 19:36:58.855965 18338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:36:58.856149 18338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:36:58.856351 18338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:36:58.858631 18338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45773 } }
I20260810 19:36:58.859128 18338 leader_election.cc:302] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92f6ad691fbd4d4f8f04548bb5428a9e; no voters:
I20260810 19:36:58.859735 18338 leader_election.cc:288] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:36:58.859911 18343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:36:58.861253 18343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [term 1 LEADER]: Becoming Leader. State: Replica: 92f6ad691fbd4d4f8f04548bb5428a9e, State: Running, Role: LEADER
I20260810 19:36:58.861855 18338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:36:58.861814 18343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45773 } }
I20260810 19:36:58.864781 18344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45773 } } }
I20260810 19:36:58.864753 18345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92f6ad691fbd4d4f8f04548bb5428a9e. Latest consensus state: current_term: 1 leader_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f6ad691fbd4d4f8f04548bb5428a9e" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 45773 } } }
I20260810 19:36:58.865219 18345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:58.865219 18344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e [sys.catalog]: This master's current role is: LEADER
I20260810 19:36:58.866215 18349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:36:58.872094 18349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:36:58.881106 18349 catalog_manager.cc:1347] Generated new cluster ID: 45f67ce722f448d0b549fec3af17a6b2
I20260810 19:36:58.881240 18349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:36:58.899386 18349 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:36:58.900729 18349 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:36:58.907115 18349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e: Generated new TSK 0
I20260810 19:36:58.907992 18349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:36:58.999754 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45773
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:59.353188 18365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:59.353485 18365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:59.353583 18365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:59.360909 18365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:59.361016 18365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:59.361128 18365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:36:59.371511 18365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:59.372802 18365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:59.374132 18365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:59.387804 18370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:59.387878 18371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:59.388334 18373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:59.388665 18365 server_base.cc:1034] running on GCE node
I20260810 19:36:59.389258 18365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:59.390195 18365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:59.391436 18365 hybrid_clock.cc:648] HybridClock initialized: now 1786390619391390 us; error 66 us; skew 500 ppm
I20260810 19:36:59.391801 18365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:59 dist-test-slave-tm1g krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390619, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:59.395045 18365 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:36:59.396539 18365 webserver.cc:460] Webserver started at http://127.29.233.129:37629/ using document root <none> and password file <none>
I20260810 19:36:59.397258 18365 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:59.397393 18365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:59.397735 18365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:59.400744 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "0561b783f2ff4a0db9399ae2233e4a60"
format_stamp: "Formatted at 2026-08-10 19:36:59 on dist-test-slave-tm1g"
server_key: "668c9cdad3e27eb2f376693c2709ebc0"
server_key_iv: "4c34ad09478efad597c61d73e4d1bd5a"
server_key_version: "encryptionkey@0"
I20260810 19:36:59.401557 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "0561b783f2ff4a0db9399ae2233e4a60"
format_stamp: "Formatted at 2026-08-10 19:36:59 on dist-test-slave-tm1g"
server_key: "668c9cdad3e27eb2f376693c2709ebc0"
server_key_iv: "4c34ad09478efad597c61d73e4d1bd5a"
server_key_version: "encryptionkey@0"
I20260810 19:36:59.406791 18365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 19:36:59.410392 18380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:59.411736 18365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:36:59.411913 18365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "0561b783f2ff4a0db9399ae2233e4a60"
format_stamp: "Formatted at 2026-08-10 19:36:59 on dist-test-slave-tm1g"
server_key: "668c9cdad3e27eb2f376693c2709ebc0"
server_key_iv: "4c34ad09478efad597c61d73e4d1bd5a"
server_key_version: "encryptionkey@0"
I20260810 19:36:59.412117 18365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:59.433633 18365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:59.437640 18365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:59.437950 18365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:59.439005 18365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:59.440351 18365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:59.440428 18365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:59.440598 18365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:59.440659 18365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:59.474594 18365 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:46487
I20260810 19:36:59.474675 18500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:46487 every 8 connection(s)
I20260810 19:36:59.476027 18365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:36:59.482734 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 18365
I20260810 19:36:59.482937 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:36:59.483412 30630 external_mini_cluster.cc:1428] Setting key 4ca6b6f0f9c85498d95c43160d23c1ea
Aug 10 19:36:59 dist-test-slave-tm1g krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390619, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:36:59.496465 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:59.478686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:57365 (local address 127.29.233.190:45773)
0810 19:36:59.479167 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 19:36:59.479171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:36:59.480057 (+ 886us) server_negotiation.cc:407] Connection header received
0810 19:36:59.481906 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:36:59.481912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:36:59.482016 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:36:59.482140 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:36:59.483854 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:59.484744 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:59.486488 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:36:59.486928 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:36:59.487076 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:59.490606 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:36:59.490611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:36:59.490614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:36:59.490649 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:36:59.493208 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:59.493977 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:59.493982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:59.493984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:59.494107 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:36:59.494628 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:36:59.494632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:36:59.494634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:36:59.494894 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:36:59.495070 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 19:36:59.495830 (+ 760us) server_negotiation.cc:299] Negotiation successful
0810 19:36:59.496084 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":260,"threads_started":1}
I20260810 19:36:59.497478 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:36:59.479167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45773 (local address 127.29.233.129:57365)
0810 19:36:59.479885 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0810 19:36:59.479925 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 19:36:59.481630 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:36:59.482413 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:36:59.482427 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:36:59.483034 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:36:59.483677 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:59.483696 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:59.484927 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:59.484939 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:59.486270 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:36:59.486283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:36:59.487170 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:36:59.487179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:36:59.487429 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:36:59.488128 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:36:59.488150 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:36:59.490385 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:36:59.493362 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:59.493370 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:59.493388 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:59.493828 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:59.494224 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:36:59.494229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:36:59.494233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:36:59.494498 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:36:59.495044 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:36:59.495053 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:36:59.495427 (+ 374us) client_negotiation.cc:769] Sending connection context
0810 19:36:59.495714 (+ 287us) client_negotiation.cc:240] Negotiation successful
0810 19:36:59.496273 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":283,"threads_started":1}
I20260810 19:36:59.499274 18501 heartbeater.cc:347] Connected to a master server at 127.29.233.190:45773
I20260810 19:36:59.499620 18501 heartbeater.cc:464] Registering TS with master...
I20260810 19:36:59.500564 18501 heartbeater.cc:511] Master 127.29.233.190:45773 requested a full tablet report, sending...
I20260810 19:36:59.502934 18296 ts_manager.cc:194] Registered new tserver with Master: 0561b783f2ff4a0db9399ae2233e4a60 (127.29.233.129:46487)
I20260810 19:36:59.504391 18296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:57365
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:36:59.557744 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45773
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:36:59.905768 18507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:36:59.906064 18507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:36:59.906162 18507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:36:59.913771 18507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:36:59.913883 18507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:36:59.913998 18507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:36:59.924212 18507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:36:59.925551 18507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:36:59.926923 18507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:36:59.940028 18517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:59.940107 18515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:36:59.940131 18514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:36:59.941231 18507 server_base.cc:1034] running on GCE node
I20260810 19:36:59.941895 18507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:36:59.942875 18507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:36:59.944080 18507 hybrid_clock.cc:648] HybridClock initialized: now 1786390619944042 us; error 26 us; skew 500 ppm
I20260810 19:36:59.944371 18507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:36:59 dist-test-slave-tm1g krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390619, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:36:59.947652 18507 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:36:59.949368 18507 webserver.cc:460] Webserver started at http://127.29.233.130:33695/ using document root <none> and password file <none>
I20260810 19:36:59.950114 18507 fs_manager.cc:359] Metadata directory not provided
I20260810 19:36:59.950250 18507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:36:59.950613 18507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:36:59.953688 18507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6"
format_stamp: "Formatted at 2026-08-10 19:36:59 on dist-test-slave-tm1g"
server_key: "2f744f34caf8106162d73ab6b05899e7"
server_key_iv: "f856ad3ebaef768116654bcbc7b1affa"
server_key_version: "encryptionkey@0"
I20260810 19:36:59.954559 18507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6"
format_stamp: "Formatted at 2026-08-10 19:36:59 on dist-test-slave-tm1g"
server_key: "2f744f34caf8106162d73ab6b05899e7"
server_key_iv: "f856ad3ebaef768116654bcbc7b1affa"
server_key_version: "encryptionkey@0"
I20260810 19:36:59.959978 18507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:36:59.963497 18524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:59.964836 18507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:36:59.965029 18507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6"
format_stamp: "Formatted at 2026-08-10 19:36:59 on dist-test-slave-tm1g"
server_key: "2f744f34caf8106162d73ab6b05899e7"
server_key_iv: "f856ad3ebaef768116654bcbc7b1affa"
server_key_version: "encryptionkey@0"
I20260810 19:36:59.965229 18507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:36:59.981879 18507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:36:59.986030 18507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:36:59.986387 18507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:36:59.987387 18507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:36:59.988737 18507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:36:59.988811 18507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:36:59.988991 18507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:36:59.989046 18507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:00.023029 18507 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:41039
I20260810 19:37:00.023077 18653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:41039 every 8 connection(s)
I20260810 19:37:00.024391 18507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:37:00.031082 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 18507
I20260810 19:37:00.031314 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:37:00.031801 30630 external_mini_cluster.cc:1428] Setting key 055e651ee0d23a4b48fd109c9a72b3cd
Aug 10 19:37:00 dist-test-slave-tm1g krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390619, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:00.049575 18657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.027553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45773 (local address 127.29.233.130:57333)
0810 19:37:00.028267 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:00.028305 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:00.030071 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:00.030905 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:00.030914 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:00.031620 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:00.032334 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.032363 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.033524 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.033531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.034986 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.034998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.036221 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.036226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.036433 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.037148 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:00.037184 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:00.039459 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:00.044054 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:00.044064 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:00.044086 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:00.044788 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:00.045318 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:00.045326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:00.045333 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:00.045611 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:00.046363 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:00.046374 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:00.046988 (+ 614us) client_negotiation.cc:769] Sending connection context
0810 19:37:00.047333 (+ 345us) client_negotiation.cc:240] Negotiation successful
0810 19:37:00.047958 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":281,"threads_started":1}
I20260810 19:37:00.051038 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.026976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:57333 (local address 127.29.233.190:45773)
0810 19:37:00.027444 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.027448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.028457 (+ 1009us) server_negotiation.cc:407] Connection header received
0810 19:37:00.030395 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.030401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.030509 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.030635 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:00.032564 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.033346 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.035149 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.035970 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.036135 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.039698 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:00.039707 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:00.039714 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:00.039761 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:00.043896 (+ 4135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:00.044961 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:00.044967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:00.044971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:00.045145 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:00.045762 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:00.045769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:00.045775 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:00.046176 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:00.046361 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.050161 (+ 3800us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.050481 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":350,"thread_start_us":237,"threads_started":1}
I20260810 19:37:00.052435 18654 heartbeater.cc:347] Connected to a master server at 127.29.233.190:45773
I20260810 19:37:00.052776 18654 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:00.053635 18654 heartbeater.cc:511] Master 127.29.233.190:45773 requested a full tablet report, sending...
I20260810 19:37:00.055096 18296 ts_manager.cc:194] Registered new tserver with Master: ea0b05c30a4b48dbb78d2798a0b3d8c6 (127.29.233.130:41039)
I20260810 19:37:00.055949 18296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:57333
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:37:00.106572 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45773
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:00.454979 18660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:00.455271 18660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:00.455459 18660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:00.463011 18660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:00.463137 18660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:00.463320 18660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:37:00.473385 18660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:45773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:00.474823 18660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:00.476222 18660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:00.489753 18669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:00.489790 18667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:00.489836 18665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:00.490665 18660 server_base.cc:1034] running on GCE node
I20260810 19:37:00.491379 18660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:00.492393 18660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:00.493649 18660 hybrid_clock.cc:648] HybridClock initialized: now 1786390620493599 us; error 42 us; skew 500 ppm
I20260810 19:37:00.494036 18660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:00 dist-test-slave-tm1g krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:00.497289 18660 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:37:00.498831 18660 webserver.cc:460] Webserver started at http://127.29.233.131:35725/ using document root <none> and password file <none>
I20260810 19:37:00.499555 18660 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:00.499660 18660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:00.499946 18660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:00.503039 18660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "7b6e6cdac52a46fe81ff589b9ad4882b"
format_stamp: "Formatted at 2026-08-10 19:37:00 on dist-test-slave-tm1g"
server_key: "d61f6380268505a715d30564a40d9644"
server_key_iv: "0876df19dfdf0fd60b9ca248e193baa0"
server_key_version: "encryptionkey@0"
I20260810 19:37:00.503786 18660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "7b6e6cdac52a46fe81ff589b9ad4882b"
format_stamp: "Formatted at 2026-08-10 19:37:00 on dist-test-slave-tm1g"
server_key: "d61f6380268505a715d30564a40d9644"
server_key_iv: "0876df19dfdf0fd60b9ca248e193baa0"
server_key_version: "encryptionkey@0"
I20260810 19:37:00.508234 18501 heartbeater.cc:503] Master 127.29.233.190:45773 was elected leader, sending a full tablet report...
I20260810 19:37:00.509574 18660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:37:00.513356 18677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:00.514803 18660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:37:00.514983 18660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "7b6e6cdac52a46fe81ff589b9ad4882b"
format_stamp: "Formatted at 2026-08-10 19:37:00 on dist-test-slave-tm1g"
server_key: "d61f6380268505a715d30564a40d9644"
server_key_iv: "0876df19dfdf0fd60b9ca248e193baa0"
server_key_version: "encryptionkey@0"
I20260810 19:37:00.515194 18660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:00.537554 18660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:00.541651 18660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:00.541975 18660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:00.543045 18660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:00.544380 18660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:00.544485 18660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:00.544631 18660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:00.544704 18660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:00.579388 18660 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:46043
I20260810 19:37:00.579452 18807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:46043 every 8 connection(s)
I20260810 19:37:00.580791 18660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:37:00.588404 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 18660
I20260810 19:37:00.588629 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:37:00.589111 30630 external_mini_cluster.cc:1428] Setting key fc3549aa0caf2f8d3ff92f4e8e27bc6e
Aug 10 19:37:00 dist-test-slave-tm1g krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:00.600693 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.583430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:37909 (local address 127.29.233.190:45773)
0810 19:37:00.583928 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.583932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.584855 (+ 923us) server_negotiation.cc:407] Connection header received
0810 19:37:00.586756 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.586763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.586861 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.586990 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:00.588452 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.589427 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.591204 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.591615 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.591770 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.595210 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:00.595215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:00.595218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:00.595251 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:00.597941 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:00.598716 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:00.598720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:00.598722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:00.598853 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:00.599223 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:00.599226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:00.599228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:00.599488 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:00.599606 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.600215 (+ 609us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.600380 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":241,"threads_started":1}
I20260810 19:37:00.601596 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.583954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:45773 (local address 127.29.233.131:37909)
0810 19:37:00.584692 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:00.584735 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:00.586487 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:00.587215 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:00.587223 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:00.587742 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:00.588283 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.588300 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.589585 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.589589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.591065 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.591080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.591857 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.591865 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.592186 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.592822 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:00.592843 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:00.595023 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:00.598102 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:00.598108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:00.598120 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:00.598609 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:00.598955 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:00.598958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:00.598960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:00.599128 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:00.599591 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:00.599596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:00.599893 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 19:37:00.600123 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 19:37:00.600493 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":263,"threads_started":1}
I20260810 19:37:00.602967 18808 heartbeater.cc:347] Connected to a master server at 127.29.233.190:45773
I20260810 19:37:00.603312 18808 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:00.604130 18808 heartbeater.cc:511] Master 127.29.233.190:45773 requested a full tablet report, sending...
I20260810 19:37:00.605540 18296 ts_manager.cc:194] Registered new tserver with Master: 7b6e6cdac52a46fe81ff589b9ad4882b (127.29.233.131:46043)
I20260810 19:37:00.606459 18296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:37909
I20260810 19:37:00.616503 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:37:00.634637 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.620796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.29.233.190:45773)
0810 19:37:00.620985 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.620991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.621434 (+ 443us) server_negotiation.cc:407] Connection header received
0810 19:37:00.621634 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.621638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.621714 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.621818 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:00.623342 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.624100 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.625419 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.625834 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.625969 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.629091 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:00.629097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:00.629100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:00.629133 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:00.631687 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:00.632488 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:00.632494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:00.632497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:00.632634 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:00.633098 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:00.633107 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:00.633109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:00.633414 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:00.633694 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.634102 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.634332 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 19:37:00.638661 18296 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:37:00.645598 18296 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:37:00.670066 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.657541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:46487 (local address 127.0.0.1:59146)
0810 19:37:00.659309 (+ 1768us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:00.659373 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:00.659603 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:00.662111 (+ 2508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:00.662116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:00.662141 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:00.662720 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.662737 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.664296 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.664303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.666576 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.666589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.668077 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.668083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.668315 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.668369 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 19:37:00.668495 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 19:37:00.668845 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1465,"mutex_wait_us":1,"thread_start_us":1257,"threads_started":1}
I20260810 19:37:00.670573 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.660146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.29.233.130:41039)
0810 19:37:00.660828 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.660840 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.661694 (+ 854us) server_negotiation.cc:407] Connection header received
0810 19:37:00.661986 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.661995 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.662217 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.662417 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:00.663431 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.664951 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.667664 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.669042 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.669206 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.669372 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.670000 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.670193 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":321,"threads_started":1}
I20260810 19:37:00.671090 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.660444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:41039 (local address 127.0.0.1:59054)
0810 19:37:00.661598 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:00.661614 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:00.661792 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:00.662686 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:00.662690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:00.662720 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:00.663229 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.663243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.665154 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.665159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.667468 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.667482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.669534 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.669540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.669741 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.669810 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 19:37:00.670517 (+ 707us) client_negotiation.cc:240] Negotiation successful
0810 19:37:00.670654 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"mutex_wait_us":25,"thread_start_us":569,"threads_started":1}
I20260810 19:37:00.671443 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.658188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:46043 (local address 127.0.0.1:57228)
0810 19:37:00.661295 (+ 3107us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:00.661315 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:00.661466 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:00.662215 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:00.662221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:00.662247 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:00.662746 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.662757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.664756 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.664761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.667362 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.667383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.669751 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.669760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.670672 (+ 912us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.670708 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 19:37:00.670803 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 19:37:00.670955 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2956,"mutex_wait_us":76,"spinlock_wait_cycles":3840,"thread_start_us":3114,"threads_started":1}
I20260810 19:37:00.671557 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.657822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59146 (local address 127.29.233.129:46487)
0810 19:37:00.658784 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.658793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.660746 (+ 1953us) server_negotiation.cc:407] Connection header received
0810 19:37:00.660883 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.660893 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.661652 (+ 759us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.662674 (+ 1022us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:00.662956 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.664159 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.666745 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.667858 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.668130 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.668364 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.670026 (+ 1662us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.671232 (+ 1206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":599,"threads_started":1}
I20260810 19:37:00.671741 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.658859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57228 (local address 127.29.233.131:46043)
0810 19:37:00.660272 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.660278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.661619 (+ 1341us) server_negotiation.cc:407] Connection header received
0810 19:37:00.661784 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.661813 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.662030 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.662218 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:00.663000 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.664595 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.667564 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.669095 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.669325 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.669525 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.671194 (+ 1669us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.671376 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261,"thread_start_us":1116,"threads_started":1}
I20260810 19:37:00.674891 18581 tablet_service.cc:1467] Processing CreateTablet for tablet 556a4a751f9c45e3be12bfe5de084347 (DEFAULT_TABLE table=test-table [id=ff4508cb63e1492dac3160f7b3a17030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:00.675185 18736 tablet_service.cc:1467] Processing CreateTablet for tablet 556a4a751f9c45e3be12bfe5de084347 (DEFAULT_TABLE table=test-table [id=ff4508cb63e1492dac3160f7b3a17030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:00.675182 18436 tablet_service.cc:1467] Processing CreateTablet for tablet 556a4a751f9c45e3be12bfe5de084347 (DEFAULT_TABLE table=test-table [id=ff4508cb63e1492dac3160f7b3a17030]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:00.676457 18581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556a4a751f9c45e3be12bfe5de084347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:00.676728 18436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556a4a751f9c45e3be12bfe5de084347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:00.676748 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556a4a751f9c45e3be12bfe5de084347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:00.691664 18828 tablet_bootstrap.cc:492] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Bootstrap starting.
I20260810 19:37:00.692621 18830 tablet_bootstrap.cc:492] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Bootstrap starting.
I20260810 19:37:00.695637 18831 tablet_bootstrap.cc:492] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Bootstrap starting.
I20260810 19:37:00.695691 18828 tablet_bootstrap.cc:654] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:00.696206 18830 tablet_bootstrap.cc:654] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:00.697017 18828 log.cc:824] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:00.697461 18830 log.cc:824] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:00.698873 18831 tablet_bootstrap.cc:654] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:00.700213 18831 log.cc:824] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:00.701290 18828 tablet_bootstrap.cc:492] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: No bootstrap required, opened a new log
I20260810 19:37:00.701381 18830 tablet_bootstrap.cc:492] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: No bootstrap required, opened a new log
I20260810 19:37:00.701586 18828 ts_tablet_manager.cc:1397] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260810 19:37:00.701691 18830 ts_tablet_manager.cc:1397] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260810 19:37:00.703382 18831 tablet_bootstrap.cc:492] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: No bootstrap required, opened a new log
I20260810 19:37:00.703636 18831 ts_tablet_manager.cc:1397] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 19:37:00.709014 18828 raft_consensus.cc:348] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.709668 18828 raft_consensus.cc:374] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:00.709834 18828 raft_consensus.cc:729] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea0b05c30a4b48dbb78d2798a0b3d8c6, State: Initialized, Role: FOLLOWER
I20260810 19:37:00.710549 18828 consensus_queue.cc:260] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.711699 18654 heartbeater.cc:503] Master 127.29.233.190:45773 was elected leader, sending a full tablet report...
I20260810 19:37:00.712419 18828 ts_tablet_manager.cc:1428] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260810 19:37:00.712592 18830 raft_consensus.cc:348] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.713083 18830 raft_consensus.cc:374] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:00.713207 18830 raft_consensus.cc:729] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0561b783f2ff4a0db9399ae2233e4a60, State: Initialized, Role: FOLLOWER
I20260810 19:37:00.713833 18830 consensus_queue.cc:260] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.714239 18831 raft_consensus.cc:348] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.715709 18830 ts_tablet_manager.cc:1428] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260810 19:37:00.716205 18831 raft_consensus.cc:374] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:00.716336 18831 raft_consensus.cc:729] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6e6cdac52a46fe81ff589b9ad4882b, State: Initialized, Role: FOLLOWER
I20260810 19:37:00.716954 18831 consensus_queue.cc:260] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.724107 18808 heartbeater.cc:503] Master 127.29.233.190:45773 was elected leader, sending a full tablet report...
I20260810 19:37:00.725183 18831 ts_tablet_manager.cc:1428] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Time spent starting tablet: real 0.021s user 0.012s sys 0.007s
W20260810 19:37:00.728717 18502 tablet.cc:2369] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:37:00.776621 18655 tablet.cc:2369] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:37:00.832512 18809 tablet.cc:2369] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:00.882786 18836 raft_consensus.cc:482] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:37:00.883109 18836 raft_consensus.cc:504] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.884862 18836 leader_election.cc:288] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea0b05c30a4b48dbb78d2798a0b3d8c6 (127.29.233.130:41039), 7b6e6cdac52a46fe81ff589b9ad4882b (127.29.233.131:46043)
I20260810 19:37:00.894791 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.885514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:60583 (local address 127.29.233.130:41039)
0810 19:37:00.885739 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.885747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.886575 (+ 828us) server_negotiation.cc:407] Connection header received
0810 19:37:00.886878 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.886884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.886967 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.887073 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:00.887947 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.889119 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.892323 (+ 3204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.893607 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.893746 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.893823 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.894371 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.894530 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:37:00.894783 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.885465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:53911 (local address 127.29.233.131:46043)
0810 19:37:00.885683 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.885690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.886569 (+ 879us) server_negotiation.cc:407] Connection header received
0810 19:37:00.886878 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.886884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.886975 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.887073 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:00.887975 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.889119 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.892322 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.893573 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.893747 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.893823 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.894371 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.894508 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 19:37:00.895010 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.885178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:41039 (local address 127.29.233.129:60583)
0810 19:37:00.886419 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:00.886444 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:00.886616 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:00.887114 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:00.887119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:00.887161 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:00.887748 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.887771 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.889320 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.889326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.892139 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.892156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.893899 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.893905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.894138 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.894191 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 19:37:00.894291 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 19:37:00.894475 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":270,"spinlock_wait_cycles":5376,"thread_start_us":740,"threads_started":1}
I20260810 19:37:00.895049 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.885260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:46043 (local address 127.29.233.129:53911)
0810 19:37:00.886452 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:00.886470 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:00.886648 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:00.887114 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:00.887119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:00.887155 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:00.887808 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.887822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.889320 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.889326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.892139 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:00.892156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.893887 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.893894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:00.894133 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.894193 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 19:37:00.894291 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 19:37:00.894475 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":301,"spinlock_wait_cycles":37632,"thread_start_us":622,"threads_started":1}
I20260810 19:37:00.896381 18759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556a4a751f9c45e3be12bfe5de084347" candidate_uuid: "0561b783f2ff4a0db9399ae2233e4a60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" is_pre_election: true
I20260810 19:37:00.896358 18603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556a4a751f9c45e3be12bfe5de084347" candidate_uuid: "0561b783f2ff4a0db9399ae2233e4a60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" is_pre_election: true
I20260810 19:37:00.896955 18759 raft_consensus.cc:2454] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0561b783f2ff4a0db9399ae2233e4a60 in term 0.
I20260810 19:37:00.896991 18603 raft_consensus.cc:2454] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0561b783f2ff4a0db9399ae2233e4a60 in term 0.
I20260810 19:37:00.897748 18384 leader_election.cc:302] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0561b783f2ff4a0db9399ae2233e4a60, 7b6e6cdac52a46fe81ff589b9ad4882b, ea0b05c30a4b48dbb78d2798a0b3d8c6; no voters:
I20260810 19:37:00.898208 18836 raft_consensus.cc:2781] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:37:00.898398 18836 raft_consensus.cc:482] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:37:00.898533 18836 raft_consensus.cc:3037] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:00.900774 18836 raft_consensus.cc:504] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.901408 18836 leader_election.cc:288] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [CANDIDATE]: Term 1 election: Requested vote from peers ea0b05c30a4b48dbb78d2798a0b3d8c6 (127.29.233.130:41039), 7b6e6cdac52a46fe81ff589b9ad4882b (127.29.233.131:46043)
I20260810 19:37:00.901942 18603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556a4a751f9c45e3be12bfe5de084347" candidate_uuid: "0561b783f2ff4a0db9399ae2233e4a60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6"
I20260810 19:37:00.901942 18759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556a4a751f9c45e3be12bfe5de084347" candidate_uuid: "0561b783f2ff4a0db9399ae2233e4a60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b"
I20260810 19:37:00.902166 18759 raft_consensus.cc:3037] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:00.902166 18603 raft_consensus.cc:3037] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:00.905483 18603 raft_consensus.cc:2454] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0561b783f2ff4a0db9399ae2233e4a60 in term 1.
I20260810 19:37:00.905483 18759 raft_consensus.cc:2454] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0561b783f2ff4a0db9399ae2233e4a60 in term 1.
I20260810 19:37:00.906049 18384 leader_election.cc:302] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0561b783f2ff4a0db9399ae2233e4a60, ea0b05c30a4b48dbb78d2798a0b3d8c6; no voters:
I20260810 19:37:00.906440 18836 raft_consensus.cc:2781] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:00.906903 18836 raft_consensus.cc:686] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 1 LEADER]: Becoming Leader. State: Replica: 0561b783f2ff4a0db9399ae2233e4a60, State: Running, Role: LEADER
I20260810 19:37:00.907450 18836 consensus_queue.cc:237] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } }
I20260810 19:37:00.912092 18295 catalog_manager.cc:5495] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0561b783f2ff4a0db9399ae2233e4a60 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "0561b783f2ff4a0db9399ae2233e4a60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0561b783f2ff4a0db9399ae2233e4a60" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 46487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 } health_report { overall_health: UNKNOWN } } }
I20260810 19:37:00.967526 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:00.962030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.29.233.129:46487)
0810 19:37:00.962282 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:00.962287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:00.962346 (+ 59us) server_negotiation.cc:407] Connection header received
0810 19:37:00.962450 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:00.962455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:00.962536 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:00.962664 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:00.963137 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.963956 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.965312 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:00.965710 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:00.965799 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:00.966293 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:00.966886 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:00.967028 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:00.967180 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 19:37:00.967248 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 19:37:00.976042 18603 raft_consensus.cc:1261] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 1 FOLLOWER]: Refusing update from remote peer 0561b783f2ff4a0db9399ae2233e4a60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:00.976042 18759 raft_consensus.cc:1261] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 1 FOLLOWER]: Refusing update from remote peer 0561b783f2ff4a0db9399ae2233e4a60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:00.977140 18836 consensus_queue.cc:1035] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea0b05c30a4b48dbb78d2798a0b3d8c6" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:00.977437 18841 consensus_queue.cc:1035] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b6e6cdac52a46fe81ff589b9ad4882b" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 46043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:00.988533 18845 mvcc.cc:205] Tried to move back new op lower bound from 7317055983507243008 to 7317055983240826880. Current Snapshot: MvccSnapshot[applied={T|T < 7317055983507243008}]
I20260810 19:37:01.003777 18295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:01.004071 18295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:01.007894 18295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92f6ad691fbd4d4f8f04548bb5428a9e: Sending DeleteTablet for 3 replicas of tablet 556a4a751f9c45e3be12bfe5de084347
I20260810 19:37:01.009094 18436 tablet_service.cc:1514] Processing DeleteTablet for tablet 556a4a751f9c45e3be12bfe5de084347 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:01 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:59146
I20260810 19:37:01.009464 18736 tablet_service.cc:1514] Processing DeleteTablet for tablet 556a4a751f9c45e3be12bfe5de084347 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:01 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:57228
I20260810 19:37:01.009500 18581 tablet_service.cc:1514] Processing DeleteTablet for tablet 556a4a751f9c45e3be12bfe5de084347 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:01 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:59054
I20260810 19:37:01.012110 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 18365
I20260810 19:37:01.021729 18855 tablet_replica.cc:330] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: stopping tablet replica
I20260810 19:37:01.038735 18855 raft_consensus.cc:2229] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:37:01.039779 18855 raft_consensus.cc:2258] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:01.042498 18855 ts_tablet_manager.cc:1905] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:01.045604 18858 tablet_replica.cc:330] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: stopping tablet replica
I20260810 19:37:01.045847 18857 tablet_replica.cc:330] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: stopping tablet replica
I20260810 19:37:01.046231 18858 raft_consensus.cc:2229] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:01.046283 18857 raft_consensus.cc:2229] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:01.046816 18857 raft_consensus.cc:2258] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:01.046818 18858 raft_consensus.cc:2258] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:01.048940 18857 ts_tablet_manager.cc:1905] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:01.049342 18858 ts_tablet_manager.cc:1905] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 19:37:01 dist-test-slave-tm1g krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:37:01.054896 18855 ts_tablet_manager.cc:1918] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:01.056190 18857 ts_tablet_manager.cc:1918] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:01.056396 18857 log.cc:1196] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/556a4a751f9c45e3be12bfe5de084347
I20260810 19:37:01.056445 18855 log.cc:1196] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/556a4a751f9c45e3be12bfe5de084347
I20260810 19:37:01.057005 18857 ts_tablet_manager.cc:1939] T 556a4a751f9c45e3be12bfe5de084347 P 7b6e6cdac52a46fe81ff589b9ad4882b: Deleting consensus metadata
I20260810 19:37:01.059054 18280 catalog_manager.cc:4841] TS 7b6e6cdac52a46fe81ff589b9ad4882b (127.29.233.131:46043): tablet 556a4a751f9c45e3be12bfe5de084347 (table test-table [id=ff4508cb63e1492dac3160f7b3a17030]) successfully deleted
I20260810 19:37:01.060012 18855 ts_tablet_manager.cc:1939] T 556a4a751f9c45e3be12bfe5de084347 P 0561b783f2ff4a0db9399ae2233e4a60: Deleting consensus metadata
I20260810 19:37:01.060135 18858 ts_tablet_manager.cc:1918] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:01.060396 18858 log.cc:1196] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/556a4a751f9c45e3be12bfe5de084347
I20260810 19:37:01.061020 18858 ts_tablet_manager.cc:1939] T 556a4a751f9c45e3be12bfe5de084347 P ea0b05c30a4b48dbb78d2798a0b3d8c6: Deleting consensus metadata
I20260810 19:37:01.061786 18282 catalog_manager.cc:4841] TS 0561b783f2ff4a0db9399ae2233e4a60 (127.29.233.129:46487): tablet 556a4a751f9c45e3be12bfe5de084347 (table test-table [id=ff4508cb63e1492dac3160f7b3a17030]) successfully deleted
I20260810 19:37:01.062137 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:01.013947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.29.233.129:46487)
0810 19:37:01.014183 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:01.014193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:01.038587 (+ 24394us) server_negotiation.cc:407] Connection header received
0810 19:37:01.039140 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:01.039146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:01.039249 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:01.039329 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:01.041368 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:01.042514 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:01.047215 (+ 4701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:01.047873 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:01.047996 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:01.054013 (+ 6017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:01.054029 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:01.054047 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:01.054103 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:01.058414 (+ 4311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:01.059368 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:01.059380 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:01.059384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:01.059584 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:01.059998 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:01.060005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:01.060009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:01.060551 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:01.060880 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:01.061501 (+ 621us) server_negotiation.cc:299] Negotiation successful
0810 19:37:01.061770 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 19:37:01.062707 18282 catalog_manager.cc:4841] TS ea0b05c30a4b48dbb78d2798a0b3d8c6 (127.29.233.130:41039): tablet 556a4a751f9c45e3be12bfe5de084347 (table test-table [id=ff4508cb63e1492dac3160f7b3a17030]) successfully deleted
I20260810 19:37:01.064190 18496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:01.162766 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 18365
I20260810 19:37:01.185528 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 18507
Aug 10 19:37:01 dist-test-slave-tm1g krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:37:01.200979 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:01.186273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.29.233.130:41039)
0810 19:37:01.186559 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:01.186564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:01.186586 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:37:01.186647 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:01.186651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:01.186715 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:01.186807 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:01.188276 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:01.189018 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:01.190449 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:01.190940 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:01.191043 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:01.195429 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:01.195445 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:01.195462 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:01.195501 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:01.198521 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:01.199140 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:01.199147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:01.199149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:01.199266 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:01.199594 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:01.199597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:01.199599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:01.199945 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:01.200205 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:01.200430 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 19:37:01.200596 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:37:01.201684 18648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:01.291992 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 18507
I20260810 19:37:01.311335 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 18660
Aug 10 19:37:01 dist-test-slave-tm1g krb5kdc[18247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:37:01.327306 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:01.312192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.29.233.131:46043)
0810 19:37:01.312408 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:01.312413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:01.312434 (+ 21us) server_negotiation.cc:407] Connection header received
0810 19:37:01.312620 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:01.312624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:01.312688 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:01.312792 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:01.314148 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:01.314954 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:01.316260 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:01.316654 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:01.316789 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:01.321668 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:01.321683 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:01.321700 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:01.321741 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:01.324420 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:01.325124 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:01.325132 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:01.325134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:01.325310 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:01.325714 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:01.325719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:01.325722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:01.326101 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:01.326543 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:01.326724 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 19:37:01.326948 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:37:01.328027 18803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:01.420311 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 18660
I20260810 19:37:01.439949 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 18264
I20260810 19:37:01.441128 18332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:01.553416 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 18264
2026-08-10T19:37:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4767 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T19:37:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:01Z Disabled control of system clock
I20260810 19:37:01.649977 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39317
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:39317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 19:37:02.003729 18868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:02.004036 18868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:02.004119 18868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:02.011485 18868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:02.011626 18868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:02.011664 18868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:02.011730 18868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:02.011775 18868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:02.021081 18868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:39317
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:02.022455 18868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:02.024384 18868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:02.037186 18873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:02.037196 18876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:02.037127 18874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:02.038280 18868 server_base.cc:1034] running on GCE node
I20260810 19:37:02.039140 18868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:02.040602 18868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:02.041831 18868 hybrid_clock.cc:648] HybridClock initialized: now 1786390622041793 us; error 35 us; skew 500 ppm
I20260810 19:37:02.042212 18868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:02.044723 18868 webserver.cc:460] Webserver started at http://127.29.233.190:38277/ using document root <none> and password file <none>
I20260810 19:37:02.045506 18868 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:02.045601 18868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:02.045934 18868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:02.049540 18868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "7339338ffc4a432692785f11e091d8e7"
format_stamp: "Formatted at 2026-08-10 19:37:02 on dist-test-slave-tm1g"
server_key: "88d3afcbadbd3bdda3f6bc16bc340a6b"
server_key_iv: "165abcc28abac69639ada412c834829a"
server_key_version: "encryptionkey@0"
I20260810 19:37:02.050473 18868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "7339338ffc4a432692785f11e091d8e7"
format_stamp: "Formatted at 2026-08-10 19:37:02 on dist-test-slave-tm1g"
server_key: "88d3afcbadbd3bdda3f6bc16bc340a6b"
server_key_iv: "165abcc28abac69639ada412c834829a"
server_key_version: "encryptionkey@0"
I20260810 19:37:02.055928 18868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 19:37:02.059408 18884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:02.060761 18868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 19:37:02.060971 18868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "7339338ffc4a432692785f11e091d8e7"
format_stamp: "Formatted at 2026-08-10 19:37:02 on dist-test-slave-tm1g"
server_key: "88d3afcbadbd3bdda3f6bc16bc340a6b"
server_key_iv: "165abcc28abac69639ada412c834829a"
server_key_version: "encryptionkey@0"
I20260810 19:37:02.061195 18868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:02.085604 18868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:02.086544 18868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:02.086869 18868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:02.106645 18868 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:39317
I20260810 19:37:02.106678 18938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:39317 every 8 connection(s)
I20260810 19:37:02.108032 18868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:02.111760 18939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:02.118036 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 18868
I20260810 19:37:02.118235 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:02.118844 30630 external_mini_cluster.cc:1428] Setting key a2f985e1879711f789dc963c961e2041
I20260810 19:37:02.123332 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7: Bootstrap starting.
I20260810 19:37:02.126533 18939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:02.127625 18939 log.cc:824] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:02.130895 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7: No bootstrap required, opened a new log
I20260810 19:37:02.133654 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:02.120933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.29.233.190:39317)
0810 19:37:02.121908 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:02.121918 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:02.121968 (+ 50us) server_negotiation.cc:407] Connection header received
0810 19:37:02.123895 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:02.123918 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:02.124376 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:02.124759 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:02.125354 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:02.126742 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:02.128469 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:02.129364 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:02.129581 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:02.130790 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:02.130802 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:02.130820 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:02.131562 (+ 742us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:02.131670 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:02.131699 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:02.131838 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:02.132025 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 19:37:02.132351 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":330,"threads_started":1}
I20260810 19:37:02.139818 18939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7339338ffc4a432692785f11e091d8e7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39317 } }
I20260810 19:37:02.140162 18939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:02.140233 18939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7339338ffc4a432692785f11e091d8e7, State: Initialized, Role: FOLLOWER
I20260810 19:37:02.140791 18939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7339338ffc4a432692785f11e091d8e7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39317 } }
I20260810 19:37:02.141041 18939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:02.141196 18939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:02.141382 18939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:02.143793 18939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7339338ffc4a432692785f11e091d8e7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39317 } }
I20260810 19:37:02.144292 18939 leader_election.cc:302] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7339338ffc4a432692785f11e091d8e7; no voters:
I20260810 19:37:02.145005 18939 leader_election.cc:288] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:02.145156 18944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:02.145907 18944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [term 1 LEADER]: Becoming Leader. State: Replica: 7339338ffc4a432692785f11e091d8e7, State: Running, Role: LEADER
I20260810 19:37:02.146468 18944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7339338ffc4a432692785f11e091d8e7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39317 } }
I20260810 19:37:02.147358 18939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:02.149369 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7339338ffc4a432692785f11e091d8e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7339338ffc4a432692785f11e091d8e7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39317 } } }
I20260810 19:37:02.149407 18946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7339338ffc4a432692785f11e091d8e7. Latest consensus state: current_term: 1 leader_uuid: "7339338ffc4a432692785f11e091d8e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7339338ffc4a432692785f11e091d8e7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39317 } } }
I20260810 19:37:02.149848 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:02.149849 18946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:02.153357 18951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:02.158906 18951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:02.168063 18951 catalog_manager.cc:1347] Generated new cluster ID: 0540cfd8b4454ca8a424850e5528089e
I20260810 19:37:02.168181 18951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:02.178661 18951 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:02.180471 18951 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:02.189003 18951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7: Generated new TSK 0
I20260810 19:37:02.190073 18951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:02.202965 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39317
--builtin_ntp_servers=127.29.233.148:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 19:37:02.552981 18964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:02.553288 18964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:02.553453 18964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:02.560925 18964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:02.561023 18964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:02.561187 18964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:37:02.570642 18964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:02.572130 18964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:02.574080 18964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:02.587751 18970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:02.587795 18973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:02.587706 18971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:02.588826 18964 server_base.cc:1034] running on GCE node
I20260810 19:37:02.589466 18964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:02.590536 18964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:02.591764 18964 hybrid_clock.cc:648] HybridClock initialized: now 1786390622591710 us; error 48 us; skew 500 ppm
I20260810 19:37:02.592041 18964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:02.594219 18964 webserver.cc:460] Webserver started at http://127.29.233.129:33771/ using document root <none> and password file <none>
I20260810 19:37:02.594959 18964 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:02.595050 18964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:02.595314 18964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:02.598109 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "ed9b29be577841e6a2a32a6250832f47"
format_stamp: "Formatted at 2026-08-10 19:37:02 on dist-test-slave-tm1g"
server_key: "d6c3355418d02bdd4479f5d6941d4712"
server_key_iv: "49183ae92bbf03cc6041e5db57c9abdc"
server_key_version: "encryptionkey@0"
I20260810 19:37:02.598929 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "ed9b29be577841e6a2a32a6250832f47"
format_stamp: "Formatted at 2026-08-10 19:37:02 on dist-test-slave-tm1g"
server_key: "d6c3355418d02bdd4479f5d6941d4712"
server_key_iv: "49183ae92bbf03cc6041e5db57c9abdc"
server_key_version: "encryptionkey@0"
I20260810 19:37:02.603993 18964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:37:02.607406 18981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:02.608716 18964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:37:02.608880 18964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "ed9b29be577841e6a2a32a6250832f47"
format_stamp: "Formatted at 2026-08-10 19:37:02 on dist-test-slave-tm1g"
server_key: "d6c3355418d02bdd4479f5d6941d4712"
server_key_iv: "49183ae92bbf03cc6041e5db57c9abdc"
server_key_version: "encryptionkey@0"
I20260810 19:37:02.609100 18964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:02.630611 18964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:02.631448 18964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:02.631771 18964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:02.632790 18964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:02.634320 18964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:02.634444 18964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:02.634588 18964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:02.634676 18964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:02.669340 18964 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:44581
I20260810 19:37:02.669435 19104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:44581 every 8 connection(s)
I20260810 19:37:02.670830 18964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:37:02.676246 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 18964
I20260810 19:37:02.676453 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:37:02.676913 30630 external_mini_cluster.cc:1428] Setting key fce91f7e32fa01f76e53dffcbe376d38
I20260810 19:37:02.679672 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39317
--builtin_ntp_servers=127.29.233.148:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 19:37:02.683912 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:02.673480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:47081 (local address 127.29.233.190:39317)
0810 19:37:02.673964 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:02.673970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:02.675237 (+ 1267us) server_negotiation.cc:407] Connection header received
0810 19:37:02.677146 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:02.677152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:02.677258 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:02.677381 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:02.678505 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:02.679273 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:02.680917 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:02.681325 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:02.681468 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:02.683027 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:02.683032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:02.683035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:02.683053 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:02.683097 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:02.683110 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:02.683214 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:02.683454 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 19:37:02.683563 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":228,"threads_started":1}
I20260810 19:37:02.684908 19108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:02.674104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39317 (local address 127.29.233.129:47081)
0810 19:37:02.675078 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:02.675120 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:02.676853 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:02.677619 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:02.677627 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:02.677718 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:02.678277 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:02.678293 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:02.679444 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:02.679449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:02.680769 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:02.680782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:02.681560 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:02.681570 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:02.681822 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:02.682672 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:37:02.682675 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:02.682716 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:37:02.682799 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:02.683214 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:02.683229 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:02.683238 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 19:37:02.683422 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 19:37:02.683833 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":354,"threads_started":1}
I20260810 19:37:02.686611 19105 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39317
I20260810 19:37:02.686949 19105 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:02.687860 19105 heartbeater.cc:511] Master 127.29.233.190:39317 requested a full tablet report, sending...
I20260810 19:37:02.690100 18901 ts_manager.cc:194] Registered new tserver with Master: ed9b29be577841e6a2a32a6250832f47 (127.29.233.129:44581)
I20260810 19:37:02.691787 18901 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.129:47081
W20260810 19:37:03.027354 19109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:03.027637 19109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:03.027722 19109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:03.035260 19109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:03.035369 19109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:03.035478 19109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:37:03.044741 19109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:03.046164 19109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:03.048004 19109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:03.061406 19114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:03.061442 19115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:03.061548 19118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:03.062207 19109 server_base.cc:1034] running on GCE node
I20260810 19:37:03.062803 19109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:03.063735 19109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:03.064971 19109 hybrid_clock.cc:648] HybridClock initialized: now 1786390623064922 us; error 38 us; skew 500 ppm
I20260810 19:37:03.065279 19109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:03.067579 19109 webserver.cc:460] Webserver started at http://127.29.233.130:41331/ using document root <none> and password file <none>
I20260810 19:37:03.068274 19109 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:03.068403 19109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:03.068714 19109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:03.071630 19109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "8026418fb1a1452abda24396311309c7"
format_stamp: "Formatted at 2026-08-10 19:37:03 on dist-test-slave-tm1g"
server_key: "e020020217c91caef44e045df30a62f0"
server_key_iv: "07455cc2c7ea37c1e38de076ac12c1f4"
server_key_version: "encryptionkey@0"
I20260810 19:37:03.072424 19109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "8026418fb1a1452abda24396311309c7"
format_stamp: "Formatted at 2026-08-10 19:37:03 on dist-test-slave-tm1g"
server_key: "e020020217c91caef44e045df30a62f0"
server_key_iv: "07455cc2c7ea37c1e38de076ac12c1f4"
server_key_version: "encryptionkey@0"
I20260810 19:37:03.077484 19109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:37:03.080937 19124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:03.082239 19109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:37:03.082479 19109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "8026418fb1a1452abda24396311309c7"
format_stamp: "Formatted at 2026-08-10 19:37:03 on dist-test-slave-tm1g"
server_key: "e020020217c91caef44e045df30a62f0"
server_key_iv: "07455cc2c7ea37c1e38de076ac12c1f4"
server_key_version: "encryptionkey@0"
I20260810 19:37:03.082743 19109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:03.109777 19109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:03.110702 19109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:03.111028 19109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:03.112082 19109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:03.113830 19109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:03.113962 19109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:03.114130 19109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:03.114229 19109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:03.152232 19109 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:43025
I20260810 19:37:03.152335 19244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:43025 every 8 connection(s)
I20260810 19:37:03.153602 19109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:37:03.153941 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 19109
I20260810 19:37:03.154119 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:37:03.154814 30630 external_mini_cluster.cc:1428] Setting key ca0a28283de33684de642e77d92048da
I20260810 19:37:03.158129 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39317
--builtin_ntp_servers=127.29.233.148:40239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 19:37:03.168906 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.156462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:44607 (local address 127.29.233.190:39317)
0810 19:37:03.156683 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.156691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.158382 (+ 1691us) server_negotiation.cc:407] Connection header received
0810 19:37:03.161298 (+ 2916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.161303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.161365 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.161453 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:03.162901 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.163617 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.166062 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.166520 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.166622 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.168205 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:03.168210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:03.168213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:03.168232 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:03.168265 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:03.168277 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:03.168348 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.168597 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.168676 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 19:37:03.170432 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.157095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39317 (local address 127.29.233.130:44607)
0810 19:37:03.158184 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:03.158239 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:03.161007 (+ 2768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:03.161720 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:03.161748 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:03.161864 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:03.162710 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.162738 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.163763 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.163771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.165914 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.165943 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.166743 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.166752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.167088 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.167846 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:37:03.167852 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:03.167911 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:37:03.168021 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:03.168373 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:03.168386 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:03.168397 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 19:37:03.168590 (+ 193us) client_negotiation.cc:240] Negotiation successful
0810 19:37:03.169008 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":318,"threads_started":1}
I20260810 19:37:03.171984 19245 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39317
I20260810 19:37:03.172310 19245 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:03.173076 19245 heartbeater.cc:511] Master 127.29.233.190:39317 requested a full tablet report, sending...
I20260810 19:37:03.174620 18901 ts_manager.cc:194] Registered new tserver with Master: 8026418fb1a1452abda24396311309c7 (127.29.233.130:43025)
I20260810 19:37:03.175899 18901 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.130:44607
W20260810 19:37:03.508816 19248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:03.509138 19248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:03.509228 19248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:03.516894 19248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:03.517009 19248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:03.517119 19248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:37:03.526362 19248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:40239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:39317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:03.527767 19248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:03.529575 19248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:03.543448 19253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:03.543443 19254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:03.543591 19256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:03.544310 19248 server_base.cc:1034] running on GCE node
I20260810 19:37:03.544930 19248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:03.545889 19248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:03.547122 19248 hybrid_clock.cc:648] HybridClock initialized: now 1786390623547079 us; error 56 us; skew 500 ppm
I20260810 19:37:03.547473 19248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:03.549921 19248 webserver.cc:460] Webserver started at http://127.29.233.131:34605/ using document root <none> and password file <none>
I20260810 19:37:03.550700 19248 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:03.550832 19248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:03.551153 19248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:03.554143 19248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "669f3038d8a14916a21e0310c65d3389"
format_stamp: "Formatted at 2026-08-10 19:37:03 on dist-test-slave-tm1g"
server_key: "ceb7f81cea72734c811127ca176d6265"
server_key_iv: "40930c926b90150a597e9137714977ad"
server_key_version: "encryptionkey@0"
I20260810 19:37:03.555037 19248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "669f3038d8a14916a21e0310c65d3389"
format_stamp: "Formatted at 2026-08-10 19:37:03 on dist-test-slave-tm1g"
server_key: "ceb7f81cea72734c811127ca176d6265"
server_key_iv: "40930c926b90150a597e9137714977ad"
server_key_version: "encryptionkey@0"
I20260810 19:37:03.560231 19248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 19:37:03.563766 19263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:03.565228 19248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 19:37:03.565436 19248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "669f3038d8a14916a21e0310c65d3389"
format_stamp: "Formatted at 2026-08-10 19:37:03 on dist-test-slave-tm1g"
server_key: "ceb7f81cea72734c811127ca176d6265"
server_key_iv: "40930c926b90150a597e9137714977ad"
server_key_version: "encryptionkey@0"
I20260810 19:37:03.565656 19248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:03.593456 19248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:03.594408 19248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:03.594759 19248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:03.595836 19248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:03.597321 19248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:03.597465 19248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:03.597592 19248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:03.597681 19248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:03.631595 19248 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:32977
I20260810 19:37:03.631661 19382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:32977 every 8 connection(s)
I20260810 19:37:03.633073 19248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:37:03.643419 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 19248
I20260810 19:37:03.643649 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:37:03.644114 30630 external_mini_cluster.cc:1428] Setting key e49dd236c0585966ab3b0de03d47484f
I20260810 19:37:03.646543 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.635678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:50371 (local address 127.29.233.190:39317)
0810 19:37:03.635861 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.635866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.637157 (+ 1291us) server_negotiation.cc:407] Connection header received
0810 19:37:03.639053 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.639060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.639155 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.639273 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:03.640648 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.641485 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.643143 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.643596 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.643745 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.645390 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:03.645432 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:03.645437 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:03.645464 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:03.645509 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:03.645535 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:03.645632 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.646044 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.646157 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:37:03.647794 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.636116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:39317 (local address 127.29.233.131:50371)
0810 19:37:03.636991 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:03.637026 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:03.638765 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:03.639535 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:03.639543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:03.639627 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:03.640300 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.640329 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.641642 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.641646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.642980 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.642992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.643849 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.643856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.644097 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.644856 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:37:03.644859 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:03.644910 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:37:03.645033 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:03.645696 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:03.645707 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:03.645716 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 19:37:03.645985 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 19:37:03.646412 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":291,"threads_started":1}
I20260810 19:37:03.649085 19383 heartbeater.cc:347] Connected to a master server at 127.29.233.190:39317
I20260810 19:37:03.649394 19383 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:03.650141 19383 heartbeater.cc:511] Master 127.29.233.190:39317 requested a full tablet report, sending...
I20260810 19:37:03.651647 18901 ts_manager.cc:194] Registered new tserver with Master: 669f3038d8a14916a21e0310c65d3389 (127.29.233.131:32977)
I20260810 19:37:03.652503 18901 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.131:50371
I20260810 19:37:03.660127 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:37:03.670935 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.664263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.29.233.190:39317)
0810 19:37:03.664481 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.664485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.664991 (+ 506us) server_negotiation.cc:407] Connection header received
0810 19:37:03.665143 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.665146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.665218 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.665318 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:03.665879 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.666720 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.668107 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.668633 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.668752 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.670090 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:03.670096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:03.670100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:03.670122 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:03.670158 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:03.670173 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:03.670391 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.670590 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.670645 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:37:03.675160 18901 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:37:03.681964 18901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:37:03.694990 19105 heartbeater.cc:503] Master 127.29.233.190:39317 was elected leader, sending a full tablet report...
I20260810 19:37:03.711107 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.694352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:44581 (local address 127.0.0.1:41860)
0810 19:37:03.695212 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:03.695262 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:03.695490 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:03.698062 (+ 2572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:03.698072 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:03.698112 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:03.699095 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.699126 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.703581 (+ 4455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.703590 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.708236 (+ 4646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.708250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.709943 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.709949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.710242 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.710330 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 19:37:03.710497 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 19:37:03.710720 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":1497,"threads_started":1}
I20260810 19:37:03.711107 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.694531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.29.233.129:44581)
0810 19:37:03.696831 (+ 2300us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.696840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.696881 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:37:03.696969 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.696976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.697230 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.699464 (+ 2234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:03.699770 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.703389 (+ 3619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.708445 (+ 5056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.709735 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.709942 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.710244 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.710556 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.710730 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2131,"thread_start_us":1995,"threads_started":1}
I20260810 19:37:03.716156 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.697675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:32977 (local address 127.0.0.1:56562)
0810 19:37:03.707234 (+ 9559us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:03.707253 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:03.707361 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:03.708808 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:03.708812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:03.708839 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:03.709215 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.709227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.711152 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.711158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.713353 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.713372 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.715354 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.715360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.715529 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.715565 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 19:37:03.715676 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 19:37:03.715811 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9450,"thread_start_us":9798,"threads_started":1}
I20260810 19:37:03.716256 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.697959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56562 (local address 127.29.233.131:32977)
0810 19:37:03.708202 (+ 10243us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.708212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.708248 (+ 36us) server_negotiation.cc:407] Connection header received
0810 19:37:03.708344 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.708352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.708596 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.708796 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:03.709398 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.710974 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.713666 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.715138 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.715341 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.715576 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.715724 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.715900 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10059,"thread_start_us":9881,"threads_started":1}
I20260810 19:37:03.716502 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.694397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:43025 (local address 127.0.0.1:38326)
0810 19:37:03.696199 (+ 1802us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:03.696221 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:03.696351 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:03.707660 (+ 11309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:03.707665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:03.707694 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:03.708172 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.708191 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.710298 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.710331 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.712596 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.712612 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.715663 (+ 3051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.715672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.715876 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.715908 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 19:37:03.715986 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 19:37:03.716117 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1635,"mutex_wait_us":1,"thread_start_us":1654,"threads_started":1}
I20260810 19:37:03.715911 19034 tablet_service.cc:1467] Processing CreateTablet for tablet e4e1e43fce7a4702ba9f5fe13089c789 (DEFAULT_TABLE table=test-table [id=299609a1cd6c4764a9fbf2970fa553e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:03.717404 19034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e1e43fce7a4702ba9f5fe13089c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:03.717700 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.695568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.29.233.130:43025)
0810 19:37:03.706964 (+ 11396us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.706987 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.707025 (+ 38us) server_negotiation.cc:407] Connection header received
0810 19:37:03.707135 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.707144 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.707434 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.708412 (+ 978us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:03.708646 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.710084 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.714014 (+ 3930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.715422 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.716913 (+ 1491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.717143 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.717209 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.717371 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11209,"thread_start_us":11077,"threads_started":1}
I20260810 19:37:03.720659 19177 tablet_service.cc:1467] Processing CreateTablet for tablet e4e1e43fce7a4702ba9f5fe13089c789 (DEFAULT_TABLE table=test-table [id=299609a1cd6c4764a9fbf2970fa553e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:03.720548 19314 tablet_service.cc:1467] Processing CreateTablet for tablet e4e1e43fce7a4702ba9f5fe13089c789 (DEFAULT_TABLE table=test-table [id=299609a1cd6c4764a9fbf2970fa553e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:03.721781 19177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e1e43fce7a4702ba9f5fe13089c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:03.721936 19314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e1e43fce7a4702ba9f5fe13089c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:03.732555 19405 tablet_bootstrap.cc:492] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Bootstrap starting.
I20260810 19:37:03.734647 19406 tablet_bootstrap.cc:492] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Bootstrap starting.
I20260810 19:37:03.736783 19407 tablet_bootstrap.cc:492] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Bootstrap starting.
I20260810 19:37:03.738003 19405 tablet_bootstrap.cc:654] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:03.738057 19406 tablet_bootstrap.cc:654] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:03.739404 19405 log.cc:824] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:03.739404 19406 log.cc:824] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:03.739884 19407 tablet_bootstrap.cc:654] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:03.741071 19407 log.cc:824] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:03.742924 19406 tablet_bootstrap.cc:492] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: No bootstrap required, opened a new log
I20260810 19:37:03.743227 19406 ts_tablet_manager.cc:1397] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 19:37:03.746866 19405 tablet_bootstrap.cc:492] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: No bootstrap required, opened a new log
I20260810 19:37:03.747176 19405 ts_tablet_manager.cc:1397] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.005s
I20260810 19:37:03.747392 19407 tablet_bootstrap.cc:492] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: No bootstrap required, opened a new log
I20260810 19:37:03.747660 19407 ts_tablet_manager.cc:1397] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260810 19:37:03.754206 19406 raft_consensus.cc:348] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.754766 19406 raft_consensus.cc:374] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:03.754877 19406 raft_consensus.cc:729] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9b29be577841e6a2a32a6250832f47, State: Initialized, Role: FOLLOWER
I20260810 19:37:03.755354 19407 raft_consensus.cc:348] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.755689 19407 raft_consensus.cc:374] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:03.755537 19406 consensus_queue.cc:260] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.755811 19407 raft_consensus.cc:729] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669f3038d8a14916a21e0310c65d3389, State: Initialized, Role: FOLLOWER
I20260810 19:37:03.756425 19407 consensus_queue.cc:260] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.757413 19406 ts_tablet_manager.cc:1428] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 19:37:03.757954 19405 raft_consensus.cc:348] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.758242 19405 raft_consensus.cc:374] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:03.758371 19405 raft_consensus.cc:729] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8026418fb1a1452abda24396311309c7, State: Initialized, Role: FOLLOWER
I20260810 19:37:03.759042 19405 consensus_queue.cc:260] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.759646 19407 ts_tablet_manager.cc:1428] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260810 19:37:03.759668 19383 heartbeater.cc:503] Master 127.29.233.190:39317 was elected leader, sending a full tablet report...
I20260810 19:37:03.760119 19245 heartbeater.cc:503] Master 127.29.233.190:39317 was elected leader, sending a full tablet report...
I20260810 19:37:03.762503 19405 ts_tablet_manager.cc:1428] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260810 19:37:03.826238 19411 raft_consensus.cc:482] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:37:03.826570 19411 raft_consensus.cc:504] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.828694 19411 leader_election.cc:288] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8026418fb1a1452abda24396311309c7 (127.29.233.130:43025), 669f3038d8a14916a21e0310c65d3389 (127.29.233.131:32977)
I20260810 19:37:03.838382 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.829242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:42969 (local address 127.29.233.131:32977)
0810 19:37:03.829422 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.829429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.830399 (+ 970us) server_negotiation.cc:407] Connection header received
0810 19:37:03.830618 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.830623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.830701 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.830832 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:03.831672 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.832860 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.836038 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.837308 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.837476 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.837537 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.837964 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.838097 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 19:37:03.838424 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.829195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:44993 (local address 127.29.233.130:43025)
0810 19:37:03.829410 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.829418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.830447 (+ 1029us) server_negotiation.cc:407] Connection header received
0810 19:37:03.830593 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.830599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.830701 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.830814 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:03.831746 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.832847 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.836038 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.837252 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.837463 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.837533 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.837964 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.838096 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:37:03.838580 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.828999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:32977 (local address 127.29.233.129:42969)
0810 19:37:03.830214 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:03.830235 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:03.830435 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:03.830845 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:03.830851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:03.830880 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:03.831435 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.831457 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.832998 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.833004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.835782 (+ 2778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.835794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.837515 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.837521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.837747 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.837815 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 19:37:03.837913 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 19:37:03.838101 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"mutex_wait_us":326,"thread_start_us":802,"threads_started":1}
I20260810 19:37:03.839040 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.829092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:43025 (local address 127.29.233.129:44993)
0810 19:37:03.830337 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:03.830352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:03.830453 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:03.830845 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:03.830851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:03.830945 (+ 94us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:03.831539 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.831553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.832999 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.833004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.835777 (+ 2773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:03.835794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.837484 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.837497 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:03.837751 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.837815 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 19:37:03.837913 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 19:37:03.838101 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"mutex_wait_us":308,"spinlock_wait_cycles":25984,"thread_start_us":522,"threads_started":1}
I20260810 19:37:03.839761 19336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e1e43fce7a4702ba9f5fe13089c789" candidate_uuid: "ed9b29be577841e6a2a32a6250832f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669f3038d8a14916a21e0310c65d3389" is_pre_election: true
I20260810 19:37:03.840060 19197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e1e43fce7a4702ba9f5fe13089c789" candidate_uuid: "ed9b29be577841e6a2a32a6250832f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8026418fb1a1452abda24396311309c7" is_pre_election: true
I20260810 19:37:03.840294 19336 raft_consensus.cc:2454] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9b29be577841e6a2a32a6250832f47 in term 0.
I20260810 19:37:03.840597 19197 raft_consensus.cc:2454] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9b29be577841e6a2a32a6250832f47 in term 0.
I20260810 19:37:03.841104 18984 leader_election.cc:302] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 669f3038d8a14916a21e0310c65d3389, ed9b29be577841e6a2a32a6250832f47; no voters:
I20260810 19:37:03.841493 19411 raft_consensus.cc:2781] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:37:03.841673 19411 raft_consensus.cc:482] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:37:03.841784 19411 raft_consensus.cc:3037] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:03.844082 19411 raft_consensus.cc:504] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.844695 19411 leader_election.cc:288] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [CANDIDATE]: Term 1 election: Requested vote from peers 8026418fb1a1452abda24396311309c7 (127.29.233.130:43025), 669f3038d8a14916a21e0310c65d3389 (127.29.233.131:32977)
I20260810 19:37:03.845201 19336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e1e43fce7a4702ba9f5fe13089c789" candidate_uuid: "ed9b29be577841e6a2a32a6250832f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669f3038d8a14916a21e0310c65d3389"
I20260810 19:37:03.845201 19197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4e1e43fce7a4702ba9f5fe13089c789" candidate_uuid: "ed9b29be577841e6a2a32a6250832f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8026418fb1a1452abda24396311309c7"
I20260810 19:37:03.845433 19336 raft_consensus.cc:3037] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:03.845428 19197 raft_consensus.cc:3037] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:03.848865 19336 raft_consensus.cc:2454] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9b29be577841e6a2a32a6250832f47 in term 1.
I20260810 19:37:03.848870 19197 raft_consensus.cc:2454] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9b29be577841e6a2a32a6250832f47 in term 1.
I20260810 19:37:03.849321 18982 leader_election.cc:302] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8026418fb1a1452abda24396311309c7, ed9b29be577841e6a2a32a6250832f47; no voters:
I20260810 19:37:03.849733 19411 raft_consensus.cc:2781] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:03.850230 19411 raft_consensus.cc:686] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 1 LEADER]: Becoming Leader. State: Replica: ed9b29be577841e6a2a32a6250832f47, State: Running, Role: LEADER
I20260810 19:37:03.850745 19411 consensus_queue.cc:237] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } }
I20260810 19:37:03.855443 18901 catalog_manager.cc:5495] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed9b29be577841e6a2a32a6250832f47 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "ed9b29be577841e6a2a32a6250832f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9b29be577841e6a2a32a6250832f47" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 44581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 } health_report { overall_health: UNKNOWN } } }
W20260810 19:37:03.884841 19385 tablet.cc:2369] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:03.902136 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.896842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.29.233.129:44581)
0810 19:37:03.897044 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.897048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.897066 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:37:03.897153 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.897157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.897247 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.897375 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:03.897783 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.898689 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.899998 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.900486 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.900603 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.901070 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:03.901605 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:03.901743 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.901880 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.901931 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 19:37:03.905907 19246 tablet.cc:2369] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:03.911006 19336 raft_consensus.cc:1261] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 1 FOLLOWER]: Refusing update from remote peer ed9b29be577841e6a2a32a6250832f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:03.911005 19197 raft_consensus.cc:1261] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 1 FOLLOWER]: Refusing update from remote peer ed9b29be577841e6a2a32a6250832f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:03.912034 19417 consensus_queue.cc:1035] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "669f3038d8a14916a21e0310c65d3389" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 32977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:03.912433 19411 consensus_queue.cc:1035] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8026418fb1a1452abda24396311309c7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 19:37:03.923421 19106 tablet.cc:2369] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:03.924705 19421 mvcc.cc:205] Tried to move back new op lower bound from 7317055995528966144 to 7317055995296579584. Current Snapshot: MvccSnapshot[applied={T|T < 7317055995528966144}]
I20260810 19:37:03.939019 18900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:03.939278 18900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:03.942966 18900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7339338ffc4a432692785f11e091d8e7: Sending DeleteTablet for 3 replicas of tablet e4e1e43fce7a4702ba9f5fe13089c789
I20260810 19:37:03.944254 19034 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e1e43fce7a4702ba9f5fe13089c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:03 UTC) from {username='slave'} at 127.0.0.1:41860
I20260810 19:37:03.944434 19314 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e1e43fce7a4702ba9f5fe13089c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:03 UTC) from {username='slave'} at 127.0.0.1:56562
I20260810 19:37:03.945035 19177 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e1e43fce7a4702ba9f5fe13089c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:03 UTC) from {username='slave'} at 127.0.0.1:38326
I20260810 19:37:03.945672 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 18964
I20260810 19:37:03.955473 19431 tablet_replica.cc:330] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: stopping tablet replica
I20260810 19:37:03.956136 19431 raft_consensus.cc:2229] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:37:03.956969 19431 raft_consensus.cc:2258] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:03.957834 19432 tablet_replica.cc:330] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: stopping tablet replica
I20260810 19:37:03.958279 19432 raft_consensus.cc:2229] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:03.958860 19432 raft_consensus.cc:2258] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:03.960322 19431 ts_tablet_manager.cc:1905] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:03.960353 19433 tablet_replica.cc:330] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: stopping tablet replica
I20260810 19:37:03.960762 19433 raft_consensus.cc:2229] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:03.961275 19433 raft_consensus.cc:2258] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:03.961922 19432 ts_tablet_manager.cc:1905] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:03.963300 19433 ts_tablet_manager.cc:1905] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:03.968230 19431 ts_tablet_manager.cc:1918] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:03.968434 19431 log.cc:1196] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/e4e1e43fce7a4702ba9f5fe13089c789
I20260810 19:37:03.969077 19431 ts_tablet_manager.cc:1939] T e4e1e43fce7a4702ba9f5fe13089c789 P ed9b29be577841e6a2a32a6250832f47: Deleting consensus metadata
I20260810 19:37:03.970953 19433 ts_tablet_manager.cc:1918] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:03.971122 19433 log.cc:1196] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/e4e1e43fce7a4702ba9f5fe13089c789
I20260810 19:37:03.971240 18887 catalog_manager.cc:4841] TS ed9b29be577841e6a2a32a6250832f47 (127.29.233.129:44581): tablet e4e1e43fce7a4702ba9f5fe13089c789 (table test-table [id=299609a1cd6c4764a9fbf2970fa553e3]) successfully deleted
I20260810 19:37:03.971715 19433 ts_tablet_manager.cc:1939] T e4e1e43fce7a4702ba9f5fe13089c789 P 8026418fb1a1452abda24396311309c7: Deleting consensus metadata
I20260810 19:37:03.972626 19432 ts_tablet_manager.cc:1918] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:03.972792 19432 log.cc:1196] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/e4e1e43fce7a4702ba9f5fe13089c789
I20260810 19:37:03.973349 19432 ts_tablet_manager.cc:1939] T e4e1e43fce7a4702ba9f5fe13089c789 P 669f3038d8a14916a21e0310c65d3389: Deleting consensus metadata
I20260810 19:37:03.973568 18885 catalog_manager.cc:4841] TS 8026418fb1a1452abda24396311309c7 (127.29.233.130:43025): tablet e4e1e43fce7a4702ba9f5fe13089c789 (table test-table [id=299609a1cd6c4764a9fbf2970fa553e3]) successfully deleted
I20260810 19:37:03.975306 18887 catalog_manager.cc:4841] TS 669f3038d8a14916a21e0310c65d3389 (127.29.233.131:32977): tablet e4e1e43fce7a4702ba9f5fe13089c789 (table test-table [id=299609a1cd6c4764a9fbf2970fa553e3]) successfully deleted
I20260810 19:37:03.981005 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:03.947600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.29.233.129:44581)
0810 19:37:03.947869 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:03.947878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:03.959786 (+ 11908us) server_negotiation.cc:407] Connection header received
0810 19:37:03.968510 (+ 8724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:03.968516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:03.968640 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:03.968759 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:03.969693 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.970929 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.973381 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:03.974049 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:03.974190 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:03.979584 (+ 5394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:03.979592 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:03.979606 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:03.980213 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:03.980262 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:03.980283 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:03.980364 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:03.980563 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 19:37:03.980650 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 19:37:03.981658 19100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:04.082352 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 18964
I20260810 19:37:04.103770 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 19109
I20260810 19:37:04.110992 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:04.104621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.29.233.130:43025)
0810 19:37:04.104830 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:04.104838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:04.104862 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:37:04.104928 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:04.104934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:04.105028 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:04.105135 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:04.105588 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:04.106503 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:04.107812 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:04.108234 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:04.108370 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:04.109522 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:04.109532 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:04.109547 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:04.110141 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:04.110190 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:04.110210 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:04.110354 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:04.110579 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 19:37:04.110696 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:37:04.111688 19240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:04.206131 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 19109
I20260810 19:37:04.226789 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 19248
I20260810 19:37:04.233848 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:04.227577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.29.233.131:32977)
0810 19:37:04.227763 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:04.227769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:04.227789 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:37:04.227961 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:04.227966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:04.228107 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:04.228213 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:04.228648 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:04.229394 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:04.230772 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:04.231175 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:04.231316 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:04.232483 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:04.232494 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:04.232509 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:04.233097 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:04.233146 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:04.233171 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:04.233295 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:04.233427 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 19:37:04.233538 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 19:37:04.234521 19378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:04.324368 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 19248
I20260810 19:37:04.344952 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 18868
I20260810 19:37:04.346134 18934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:04.457562 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 18868
2026-08-10T19:37:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2903 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:04 dist-test-slave-tm1g krb5kdc[19441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:04 dist-test-slave-tm1g krb5kdc[19441](info): set up 2 sockets
Aug 10 19:37:04 dist-test-slave-tm1g krb5kdc[19441](info): commencing operation
krb5kdc: starting...
W20260810 19:37:05.635399 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:05 dist-test-slave-tm1g krb5kdc[19441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:05.957345 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:38757
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:38757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:06.303201 19455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:06.303583 19455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:06.303743 19455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:06.311069 19455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:06.311215 19455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:06.311332 19455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:06.311440 19455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:06.311517 19455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:06.321538 19455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:38757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:38757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:06.322999 19455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:06.324376 19455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:06.336917 19460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:06.337069 19461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:06.337010 19464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:06.338124 19455 server_base.cc:1034] running on GCE node
I20260810 19:37:06.339068 19455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:06.340605 19455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:06.341910 19455 hybrid_clock.cc:648] HybridClock initialized: now 1786390626341869 us; error 45 us; skew 500 ppm
I20260810 19:37:06.342360 19455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:06 dist-test-slave-tm1g krb5kdc[19441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:06.345531 19455 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:06.347242 19455 webserver.cc:460] Webserver started at http://127.29.233.190:35139/ using document root <none> and password file <none>
I20260810 19:37:06.347970 19455 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:06.348125 19455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:06.348451 19455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:06.351608 19455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "58a3fb8111c348b79390f2d9430d4612"
format_stamp: "Formatted at 2026-08-10 19:37:06 on dist-test-slave-tm1g"
server_key: "281251cacefee7f7aa724b2e76a275a8"
server_key_iv: "382a078490a8730f85ea231b21e3d85d"
server_key_version: "encryptionkey@0"
I20260810 19:37:06.352398 19455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "58a3fb8111c348b79390f2d9430d4612"
format_stamp: "Formatted at 2026-08-10 19:37:06 on dist-test-slave-tm1g"
server_key: "281251cacefee7f7aa724b2e76a275a8"
server_key_iv: "382a078490a8730f85ea231b21e3d85d"
server_key_version: "encryptionkey@0"
I20260810 19:37:06.357741 19455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:37:06.361264 19472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:06.362572 19455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 19:37:06.362766 19455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "58a3fb8111c348b79390f2d9430d4612"
format_stamp: "Formatted at 2026-08-10 19:37:06 on dist-test-slave-tm1g"
server_key: "281251cacefee7f7aa724b2e76a275a8"
server_key_iv: "382a078490a8730f85ea231b21e3d85d"
server_key_version: "encryptionkey@0"
I20260810 19:37:06.362968 19455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:06.389402 19455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:06.393533 19455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:06.393836 19455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:06.413612 19455 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:38757
I20260810 19:37:06.413637 19531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:38757 every 8 connection(s)
I20260810 19:37:06.415091 19455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:06.418557 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 19455
I20260810 19:37:06.418754 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:06.419085 19532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:06.419334 30630 external_mini_cluster.cc:1428] Setting key 02387be0e4d4cddd805861045c885f82
I20260810 19:37:06.434154 19532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612: Bootstrap starting.
Aug 10 19:37:06 dist-test-slave-tm1g krb5kdc[19441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:06.437410 19532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:06.438572 19532 log.cc:824] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:06.441425 19532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612: No bootstrap required, opened a new log
I20260810 19:37:06.443123 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:06.421179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.29.233.190:38757)
0810 19:37:06.422089 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:06.422105 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:06.422176 (+ 71us) server_negotiation.cc:407] Connection header received
0810 19:37:06.423940 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:06.423961 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:06.424328 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:06.424674 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:06.426708 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:06.428013 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:06.429919 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:06.430772 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:06.430912 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:06.435842 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:06.435850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:06.435863 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:06.435903 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:06.439100 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:06.439855 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:06.439862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:06.439868 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:06.440032 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:06.440422 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:06.440427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:06.440432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:06.440938 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:06.441233 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:06.441684 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 19:37:06.442113 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":414,"threads_started":1}
I20260810 19:37:06.449501 19532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3fb8111c348b79390f2d9430d4612" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38757 } }
I20260810 19:37:06.449939 19532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:06.450065 19532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a3fb8111c348b79390f2d9430d4612, State: Initialized, Role: FOLLOWER
I20260810 19:37:06.450717 19532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3fb8111c348b79390f2d9430d4612" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38757 } }
I20260810 19:37:06.450981 19532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:06.451126 19532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:06.451318 19532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:06.453495 19532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3fb8111c348b79390f2d9430d4612" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38757 } }
I20260810 19:37:06.453997 19532 leader_election.cc:302] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58a3fb8111c348b79390f2d9430d4612; no voters:
I20260810 19:37:06.454667 19532 leader_election.cc:288] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:06.454803 19538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:06.455508 19538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [term 1 LEADER]: Becoming Leader. State: Replica: 58a3fb8111c348b79390f2d9430d4612, State: Running, Role: LEADER
I20260810 19:37:06.456055 19538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3fb8111c348b79390f2d9430d4612" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38757 } }
I20260810 19:37:06.456806 19532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:06.459332 19539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58a3fb8111c348b79390f2d9430d4612" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3fb8111c348b79390f2d9430d4612" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38757 } } }
I20260810 19:37:06.459400 19541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a3fb8111c348b79390f2d9430d4612. Latest consensus state: current_term: 1 leader_uuid: "58a3fb8111c348b79390f2d9430d4612" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a3fb8111c348b79390f2d9430d4612" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38757 } } }
I20260810 19:37:06.459736 19539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:06.459820 19541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:06.462028 19547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:06.468549 19547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:06.476876 19547 catalog_manager.cc:1347] Generated new cluster ID: 8798703d11364e0e9289cf6a35e963ef
I20260810 19:37:06.477003 19547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:06.492944 19547 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:06.494606 19547 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:06.505918 19547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612: Generated new TSK 0
I20260810 19:37:06.507062 19547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.233.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
Entry for principal kudu/127.29.233.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab.
I20260810 19:37:06.592326 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:38757
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:06.945335 19565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:06.945669 19565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:06.945840 19565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:06.953476 19565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:06.953634 19565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:06.953820 19565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:37:06.964140 19565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.129.keytab
--principal=kudu/127.29.233.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:38757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:06.965467 19565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:06.966893 19565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:06.980916 19571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:06.981026 19572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:06.980981 19574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:06.982576 19565 server_base.cc:1034] running on GCE node
I20260810 19:37:06.983143 19565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:06.984066 19565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:06.985319 19565 hybrid_clock.cc:648] HybridClock initialized: now 1786390626985265 us; error 54 us; skew 500 ppm
I20260810 19:37:06.985730 19565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:06 dist-test-slave-tm1g krb5kdc[19441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:06.989123 19565 init.cc:377] Logged in from keytab as kudu/127.29.233.129@KRBTEST.COM (short username kudu)
I20260810 19:37:06.990754 19565 webserver.cc:460] Webserver started at http://127.29.233.129:39203/ using document root <none> and password file <none>
I20260810 19:37:06.991489 19565 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:06.991641 19565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:06.991978 19565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:06.995014 19565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "4c2be8f09200444baa06193b692b7b30"
format_stamp: "Formatted at 2026-08-10 19:37:06 on dist-test-slave-tm1g"
server_key: "23294d0e4477a16f8be6a67f1f449a07"
server_key_iv: "d488d9f297558f8dc3901aae08aa00d7"
server_key_version: "encryptionkey@0"
I20260810 19:37:06.995800 19565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "4c2be8f09200444baa06193b692b7b30"
format_stamp: "Formatted at 2026-08-10 19:37:06 on dist-test-slave-tm1g"
server_key: "23294d0e4477a16f8be6a67f1f449a07"
server_key_iv: "d488d9f297558f8dc3901aae08aa00d7"
server_key_version: "encryptionkey@0"
I20260810 19:37:07.001048 19565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:37:07.004735 19581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:07.006371 19565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:37:07.006515 19565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "4c2be8f09200444baa06193b692b7b30"
format_stamp: "Formatted at 2026-08-10 19:37:06 on dist-test-slave-tm1g"
server_key: "23294d0e4477a16f8be6a67f1f449a07"
server_key_iv: "d488d9f297558f8dc3901aae08aa00d7"
server_key_version: "encryptionkey@0"
I20260810 19:37:07.006680 19565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:07.023232 19565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:07.027684 19565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:07.027962 19565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:07.028909 19565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:07.030332 19565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:07.030421 19565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:07.030520 19565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:07.030565 19565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:07.065567 19565 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:40223
I20260810 19:37:07.065675 19697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:40223 every 8 connection(s)
I20260810 19:37:07.067066 19565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:37:07.067615 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 19565
I20260810 19:37:07.067844 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:37:07.068501 30630 external_mini_cluster.cc:1428] Setting key 090367246e5d8b45a1cc8c55356eb02d
Aug 10 19:37:07 dist-test-slave-tm1g krb5kdc[19441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.129@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:07.092716 19701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:07.070228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:49153 (local address 127.29.233.190:38757)
0810 19:37:07.072925 (+ 2697us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:07.072930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:07.072949 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:37:07.075216 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:07.075220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:07.075289 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:07.075395 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:07.077602 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.078462 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.081038 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.081489 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.081600 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:07.086278 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:07.086283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:07.086286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:07.086416 (+ 130us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:07.089108 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:07.090100 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:07.090104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:07.090106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:07.090241 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:07.090873 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:07.090876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:07.090878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:07.091138 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:07.091243 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:07.092259 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0810 19:37:07.092450 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2594,"thread_start_us":2473,"threads_started":1}
I20260810 19:37:07.094444 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:07.071046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:38757 (local address 127.29.233.129:49153)
0810 19:37:07.072065 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:07.072117 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:07.074879 (+ 2762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:07.075693 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:07.075708 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:07.076545 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:07.077399 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:07.077427 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.078639 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.078647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:07.080866 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:07.080885 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.081744 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.081753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:07.082101 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:07.082908 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:07.082944 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:07.086019 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:07.089270 (+ 3251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:07.089283 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:07.089304 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:07.089961 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:07.090473 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:07.090479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:07.090485 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:07.090757 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:07.091282 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:07.091293 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:07.091794 (+ 501us) client_negotiation.cc:769] Sending connection context
0810 19:37:07.092196 (+ 402us) client_negotiation.cc:240] Negotiation successful
0810 19:37:07.092794 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":395,"threads_started":1}
I20260810 19:37:07.096149 19699 heartbeater.cc:347] Connected to a master server at 127.29.233.190:38757
I20260810 19:37:07.096467 19699 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:07.097203 19699 heartbeater.cc:511] Master 127.29.233.190:38757 requested a full tablet report, sending...
I20260810 19:37:07.099385 19493 ts_manager.cc:194] Registered new tserver with Master: 4c2be8f09200444baa06193b692b7b30 (127.29.233.129:40223)
I20260810 19:37:07.100894 19493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.129@KRBTEST.COM'} at 127.29.233.129:49153
WARNING: no policy specified for kudu/127.29.233.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
Entry for principal kudu/127.29.233.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab.
I20260810 19:37:07.145455 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:38757
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:07.496721 19707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:07.497059 19707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:07.497222 19707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:07.504750 19707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:07.504904 19707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:07.505079 19707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:37:07.515162 19707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.130.keytab
--principal=kudu/127.29.233.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:38757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:07.516556 19707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:07.517982 19707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:07.531697 19713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:07.531659 19712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:07.531659 19715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:07.532567 19707 server_base.cc:1034] running on GCE node
I20260810 19:37:07.533214 19707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:07.534183 19707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:07.535413 19707 hybrid_clock.cc:648] HybridClock initialized: now 1786390627535384 us; error 71 us; skew 500 ppm
I20260810 19:37:07.535751 19707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:07 dist-test-slave-tm1g krb5kdc[19441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390627, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:07.539100 19707 init.cc:377] Logged in from keytab as kudu/127.29.233.130@KRBTEST.COM (short username kudu)
I20260810 19:37:07.540555 19707 webserver.cc:460] Webserver started at http://127.29.233.130:34497/ using document root <none> and password file <none>
I20260810 19:37:07.541271 19707 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:07.541419 19707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:07.541749 19707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:07.544692 19707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "b734dc0348db42558a176fa0e13a1314"
format_stamp: "Formatted at 2026-08-10 19:37:07 on dist-test-slave-tm1g"
server_key: "92aebff7c02d8a72b681de0f3bbd8278"
server_key_iv: "e3f4f763fd6241bbf453bad61fed5649"
server_key_version: "encryptionkey@0"
I20260810 19:37:07.545485 19707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "b734dc0348db42558a176fa0e13a1314"
format_stamp: "Formatted at 2026-08-10 19:37:07 on dist-test-slave-tm1g"
server_key: "92aebff7c02d8a72b681de0f3bbd8278"
server_key_iv: "e3f4f763fd6241bbf453bad61fed5649"
server_key_version: "encryptionkey@0"
I20260810 19:37:07.550607 19707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 19:37:07.553969 19722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:07.555294 19707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:37:07.555472 19707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "b734dc0348db42558a176fa0e13a1314"
format_stamp: "Formatted at 2026-08-10 19:37:07 on dist-test-slave-tm1g"
server_key: "92aebff7c02d8a72b681de0f3bbd8278"
server_key_iv: "e3f4f763fd6241bbf453bad61fed5649"
server_key_version: "encryptionkey@0"
I20260810 19:37:07.555685 19707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:07.593516 19707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:07.597970 19707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:07.598287 19707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:07.599349 19707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:07.600666 19707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:07.600775 19707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:07.600941 19707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:07.601016 19707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:07.635573 19707 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:39433
I20260810 19:37:07.635609 19846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:39433 every 8 connection(s)
I20260810 19:37:07.636999 19707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:37:07.639094 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 19707
I20260810 19:37:07.639289 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:37:07.639897 30630 external_mini_cluster.cc:1428] Setting key b88495ddea07a0589cabf4251197a852
Aug 10 19:37:07 dist-test-slave-tm1g krb5kdc[19441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390627, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.130@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:07.656993 19850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:07.639711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:49615 (local address 127.29.233.190:38757)
0810 19:37:07.640218 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:07.640226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:07.641271 (+ 1045us) server_negotiation.cc:407] Connection header received
0810 19:37:07.643180 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:07.643185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:07.643275 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:07.643391 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:07.644849 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.645600 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.647176 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.647604 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.647742 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:07.651254 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:07.651259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:07.651262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:07.651294 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:07.653852 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:07.654624 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:07.654631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:07.654640 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:07.654772 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:07.655208 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:07.655213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:07.655215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:07.655479 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:07.655641 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:07.656315 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 19:37:07.656551 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":257,"threads_started":1}
I20260810 19:37:07.657931 19851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:07.640271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:38757 (local address 127.29.233.130:49615)
0810 19:37:07.641111 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:07.641156 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:07.642924 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:07.643629 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:07.643637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:07.644142 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:07.644675 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:07.644691 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.645762 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.645766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:07.647033 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:07.647044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:07.647831 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:07.647840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:07.648065 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:07.648812 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:07.648844 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:07.651027 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:07.653996 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:07.654005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:07.654020 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:07.654497 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:07.654892 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:07.654897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:07.654901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:07.655079 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:07.655608 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:07.655616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:07.655940 (+ 324us) client_negotiation.cc:769] Sending connection context
0810 19:37:07.656203 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 19:37:07.656717 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":370,"threads_started":1}
I20260810 19:37:07.659498 19847 heartbeater.cc:347] Connected to a master server at 127.29.233.190:38757
I20260810 19:37:07.659811 19847 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:07.660645 19847 heartbeater.cc:511] Master 127.29.233.190:38757 requested a full tablet report, sending...
I20260810 19:37:07.662132 19493 ts_manager.cc:194] Registered new tserver with Master: b734dc0348db42558a176fa0e13a1314 (127.29.233.130:39433)
I20260810 19:37:07.663497 19493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.130@KRBTEST.COM'} at 127.29.233.130:49615
WARNING: no policy specified for kudu/127.29.233.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
Entry for principal kudu/127.29.233.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab.
I20260810 19:37:07.714021 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:38757
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:08.074756 19854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:08.075016 19854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:08.075176 19854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:08.082392 19854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:08.082495 19854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:08.082659 19854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:37:08.092263 19854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.131.keytab
--principal=kudu/127.29.233.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:38757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:08.093662 19854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:08.095005 19854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 19:37:08.104187 19699 heartbeater.cc:503] Master 127.29.233.190:38757 was elected leader, sending a full tablet report...
W20260810 19:37:08.108456 19860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:08.108582 19863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:08.108608 19861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:08.109541 19854 server_base.cc:1034] running on GCE node
I20260810 19:37:08.110224 19854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:08.111323 19854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:08.112529 19854 hybrid_clock.cc:648] HybridClock initialized: now 1786390628112465 us; error 53 us; skew 500 ppm
I20260810 19:37:08.112861 19854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:08 dist-test-slave-tm1g krb5kdc[19441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390628, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:08.116034 19854 init.cc:377] Logged in from keytab as kudu/127.29.233.131@KRBTEST.COM (short username kudu)
I20260810 19:37:08.117599 19854 webserver.cc:460] Webserver started at http://127.29.233.131:46715/ using document root <none> and password file <none>
I20260810 19:37:08.118366 19854 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:08.118516 19854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:08.118849 19854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:08.121817 19854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "93c1edb72250444c9bcef19592c06718"
format_stamp: "Formatted at 2026-08-10 19:37:08 on dist-test-slave-tm1g"
server_key: "82674188457773a6e07f3e229153cd91"
server_key_iv: "d4273058ba7523f48857d5f227b5105b"
server_key_version: "encryptionkey@0"
I20260810 19:37:08.122659 19854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "93c1edb72250444c9bcef19592c06718"
format_stamp: "Formatted at 2026-08-10 19:37:08 on dist-test-slave-tm1g"
server_key: "82674188457773a6e07f3e229153cd91"
server_key_iv: "d4273058ba7523f48857d5f227b5105b"
server_key_version: "encryptionkey@0"
I20260810 19:37:08.127969 19854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:37:08.131556 19871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:08.132792 19854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 19:37:08.132957 19854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "93c1edb72250444c9bcef19592c06718"
format_stamp: "Formatted at 2026-08-10 19:37:08 on dist-test-slave-tm1g"
server_key: "82674188457773a6e07f3e229153cd91"
server_key_iv: "d4273058ba7523f48857d5f227b5105b"
server_key_version: "encryptionkey@0"
I20260810 19:37:08.133152 19854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:08.151994 19854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:08.155958 19854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:08.156262 19854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:08.157253 19854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:08.158666 19854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:08.158771 19854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:08.158936 19854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:08.159010 19854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:08.194038 19854 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:42263
I20260810 19:37:08.194092 19989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:42263 every 8 connection(s)
I20260810 19:37:08.195379 19854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:37:08.202701 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 19854
I20260810 19:37:08.202911 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:37:08.203379 30630 external_mini_cluster.cc:1428] Setting key a84d6ba26f5d598cca551408bb79e7bb
Aug 10 19:37:08 dist-test-slave-tm1g krb5kdc[19441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390628, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.131@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:08.215235 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.197842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:39111 (local address 127.29.233.190:38757)
0810 19:37:08.198374 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.198382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.199308 (+ 926us) server_negotiation.cc:407] Connection header received
0810 19:37:08.201064 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.201068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.201137 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.201234 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:08.202736 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.203729 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.205413 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.206042 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.206173 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.209496 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:08.209501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:08.209504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:08.209540 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:08.212425 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.213081 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.213084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.213086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.213211 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.213626 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.213630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.213632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.213937 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:08.214068 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.214760 (+ 692us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.214945 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":227,"threads_started":1}
I20260810 19:37:08.216207 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.198445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:38757 (local address 127.29.233.131:39111)
0810 19:37:08.199142 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.199180 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:08.200798 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:08.201431 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:08.201439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:08.201959 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:08.202551 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.202569 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.203897 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.203901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.205258 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.205270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.206267 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.206272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.206571 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.207189 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:08.207215 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:08.209303 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:08.212545 (+ 3242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:08.212551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:08.212564 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:08.212974 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:08.213304 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:08.213308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:08.213310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:08.213484 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:08.214050 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:08.214055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:08.214423 (+ 368us) client_negotiation.cc:769] Sending connection context
0810 19:37:08.214659 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 19:37:08.215044 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":263,"threads_started":1}
I20260810 19:37:08.217885 19990 heartbeater.cc:347] Connected to a master server at 127.29.233.190:38757
I20260810 19:37:08.218266 19990 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:08.219255 19990 heartbeater.cc:511] Master 127.29.233.190:38757 requested a full tablet report, sending...
I20260810 19:37:08.220876 19492 ts_manager.cc:194] Registered new tserver with Master: 93c1edb72250444c9bcef19592c06718 (127.29.233.131:42263)
I20260810 19:37:08.221798 19492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.233.131@KRBTEST.COM'} at 127.29.233.131:39111
I20260810 19:37:08.231283 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:37:08.248387 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.235423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.29.233.190:38757)
0810 19:37:08.235641 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.235649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.236032 (+ 383us) server_negotiation.cc:407] Connection header received
0810 19:37:08.236217 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.236224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.236298 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.236406 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:08.237944 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.238764 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.240065 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.240505 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.240642 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.243183 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:08.243188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:08.243192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:08.243224 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:08.245712 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.246399 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.246405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.246409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.246541 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.246917 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.246925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.246929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.247245 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:08.247507 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.247845 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.248072 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:37:08.252489 19492 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:37:08.259642 19492 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:37:08.284838 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.272575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:42263 (local address 127.0.0.1:33798)
0810 19:37:08.273938 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.273985 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:08.274196 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:08.277490 (+ 3294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:08.277497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:08.277546 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:08.278017 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.278032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.280122 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.280128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.281889 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.281904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.283782 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.283788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.284000 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.284062 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 19:37:08.284206 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 19:37:08.284417 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"mutex_wait_us":60,"thread_start_us":1860,"threads_started":1}
I20260810 19:37:08.284842 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.272872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33798 (local address 127.29.233.131:42263)
0810 19:37:08.276987 (+ 4115us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.276993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.277026 (+ 33us) server_negotiation.cc:407] Connection header received
0810 19:37:08.277100 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.277105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.277285 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.277484 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:08.278254 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.279975 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.282068 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.283548 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.283749 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.283915 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.284327 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.284489 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3981,"thread_start_us":3833,"threads_started":1}
I20260810 19:37:08.286618 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.272819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:44404)
0810 19:37:08.276361 (+ 3542us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.276379 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:08.276492 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:08.278858 (+ 2366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:08.278863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:08.278889 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:08.279342 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.279354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.281481 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.281487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.283608 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.283646 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.285757 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.285763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.285928 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.285958 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 19:37:08.286103 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 19:37:08.286229 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3422,"thread_start_us":5431,"threads_started":1}
I20260810 19:37:08.286679 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.273510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44404 (local address 127.29.233.129:40223)
0810 19:37:08.278253 (+ 4743us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.278260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.278286 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:37:08.278412 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.278421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.278636 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.279541 (+ 905us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:08.279764 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.281333 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.283851 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.285524 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.285740 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.285908 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.286146 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.286331 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4566,"thread_start_us":4370,"threads_started":1}
I20260810 19:37:08.286988 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.273774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36552 (local address 127.29.233.130:39433)
0810 19:37:08.275938 (+ 2164us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.275946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.278683 (+ 2737us) server_negotiation.cc:407] Connection header received
0810 19:37:08.279015 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.279022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.279227 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.279404 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:08.280177 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.281610 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.284240 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.285649 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.285811 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.285973 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.286521 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.286689 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2016,"thread_start_us":1923,"threads_started":1}
I20260810 19:37:08.288093 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.273219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:39433 (local address 127.0.0.1:36552)
0810 19:37:08.278539 (+ 5320us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.278577 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:08.278793 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:08.279685 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:08.279690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:08.279716 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:08.280017 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.280024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.281755 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.281761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.284085 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.284098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.286083 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.286090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.286265 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.286321 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 19:37:08.287556 (+ 1235us) client_negotiation.cc:240] Negotiation successful
0810 19:37:08.287690 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5177,"mutex_wait_us":2,"thread_start_us":3313,"threads_started":1}
I20260810 19:37:08.289511 19922 tablet_service.cc:1467] Processing CreateTablet for tablet c85b7b4f586b4d99ab60dd896bffceac (DEFAULT_TABLE table=test-table [id=9e6c7b87c58a4257a60e7840abd660f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:08.289954 19631 tablet_service.cc:1467] Processing CreateTablet for tablet c85b7b4f586b4d99ab60dd896bffceac (DEFAULT_TABLE table=test-table [id=9e6c7b87c58a4257a60e7840abd660f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:08.290927 19922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85b7b4f586b4d99ab60dd896bffceac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:08.291376 19631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85b7b4f586b4d99ab60dd896bffceac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:08.291774 19776 tablet_service.cc:1467] Processing CreateTablet for tablet c85b7b4f586b4d99ab60dd896bffceac (DEFAULT_TABLE table=test-table [id=9e6c7b87c58a4257a60e7840abd660f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:08.293143 19776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85b7b4f586b4d99ab60dd896bffceac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:08.302539 20012 tablet_bootstrap.cc:492] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Bootstrap starting.
I20260810 19:37:08.305656 20012 tablet_bootstrap.cc:654] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:08.307379 20012 log.cc:824] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:08.308287 20014 tablet_bootstrap.cc:492] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Bootstrap starting.
I20260810 19:37:08.308648 20013 tablet_bootstrap.cc:492] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Bootstrap starting.
I20260810 19:37:08.311513 20012 tablet_bootstrap.cc:492] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: No bootstrap required, opened a new log
I20260810 19:37:08.311798 20012 ts_tablet_manager.cc:1397] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 19:37:08.311928 20014 tablet_bootstrap.cc:654] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:08.312140 20013 tablet_bootstrap.cc:654] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:08.313004 20014 log.cc:824] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:08.313331 20013 log.cc:824] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:08.316586 20013 tablet_bootstrap.cc:492] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: No bootstrap required, opened a new log
I20260810 19:37:08.316792 20014 tablet_bootstrap.cc:492] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: No bootstrap required, opened a new log
I20260810 19:37:08.316881 20013 ts_tablet_manager.cc:1397] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260810 19:37:08.317056 20014 ts_tablet_manager.cc:1397] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 19:37:08.320652 20012 raft_consensus.cc:348] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.321040 20012 raft_consensus.cc:374] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:08.321167 20012 raft_consensus.cc:729] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c1edb72250444c9bcef19592c06718, State: Initialized, Role: FOLLOWER
I20260810 19:37:08.321794 20012 consensus_queue.cc:260] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.322988 19990 heartbeater.cc:503] Master 127.29.233.190:38757 was elected leader, sending a full tablet report...
I20260810 19:37:08.323508 20012 ts_tablet_manager.cc:1428] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 19:37:08.327859 20013 raft_consensus.cc:348] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.328325 20013 raft_consensus.cc:374] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:08.328436 20013 raft_consensus.cc:729] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2be8f09200444baa06193b692b7b30, State: Initialized, Role: FOLLOWER
I20260810 19:37:08.328771 20014 raft_consensus.cc:348] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.329234 20014 raft_consensus.cc:374] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:08.329121 20013 consensus_queue.cc:260] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.329342 20014 raft_consensus.cc:729] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b734dc0348db42558a176fa0e13a1314, State: Initialized, Role: FOLLOWER
I20260810 19:37:08.329936 20014 consensus_queue.cc:260] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.331110 20013 ts_tablet_manager.cc:1428] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 19:37:08.331110 19847 heartbeater.cc:503] Master 127.29.233.190:38757 was elected leader, sending a full tablet report...
I20260810 19:37:08.331743 20014 ts_tablet_manager.cc:1428] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
W20260810 19:37:08.389102 19848 tablet.cc:2369] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:37:08.447118 19991 tablet.cc:2369] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:08.549019 20019 raft_consensus.cc:482] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:37:08.549388 20019 raft_consensus.cc:504] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.551362 20019 leader_election.cc:288] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c1edb72250444c9bcef19592c06718 (127.29.233.131:42263), b734dc0348db42558a176fa0e13a1314 (127.29.233.130:39433)
I20260810 19:37:08.558817 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.551765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:58791 (local address 127.29.233.131:42263)
0810 19:37:08.551985 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.551992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.552728 (+ 736us) server_negotiation.cc:407] Connection header received
0810 19:37:08.552956 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.552962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.553046 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.553129 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:08.553816 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.554852 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.556873 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.557692 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.557914 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.557992 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.558384 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.558522 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:37:08.558826 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.552021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:48407 (local address 127.29.233.130:39433)
0810 19:37:08.552254 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.552259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.552939 (+ 680us) server_negotiation.cc:407] Connection header received
0810 19:37:08.553089 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.553093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.553186 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.553286 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:08.553974 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.554857 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.556767 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.557670 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.557816 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.557907 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.558387 (+ 480us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.558543 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 19:37:08.558997 20022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.551873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:39433 (local address 127.29.233.129:48407)
0810 19:37:08.552795 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.552809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:08.552960 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:08.553318 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:08.553322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:08.553350 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:08.553809 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.553821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.554994 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.555001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.556594 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.556606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.557966 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.557974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.558143 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.558230 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 19:37:08.558346 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 19:37:08.558574 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":141,"thread_start_us":612,"threads_started":1}
I20260810 19:37:08.559464 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.551590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:42263 (local address 127.29.233.129:58791)
0810 19:37:08.552579 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.552608 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:08.552791 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:08.553173 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:08.553179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:08.553203 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:08.553647 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.553658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.554989 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.554996 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.556631 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:08.556644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.558040 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.558046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:08.558190 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.558226 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 19:37:08.558359 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 19:37:08.558569 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":103,"thread_start_us":584,"threads_started":1}
I20260810 19:37:08.560137 19799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b7b4f586b4d99ab60dd896bffceac" candidate_uuid: "4c2be8f09200444baa06193b692b7b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b734dc0348db42558a176fa0e13a1314" is_pre_election: true
I20260810 19:37:08.560480 19944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b7b4f586b4d99ab60dd896bffceac" candidate_uuid: "4c2be8f09200444baa06193b692b7b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c1edb72250444c9bcef19592c06718" is_pre_election: true
I20260810 19:37:08.560850 19799 raft_consensus.cc:2454] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2be8f09200444baa06193b692b7b30 in term 0.
I20260810 19:37:08.561053 19944 raft_consensus.cc:2454] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2be8f09200444baa06193b692b7b30 in term 0.
I20260810 19:37:08.561610 19582 leader_election.cc:302] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2be8f09200444baa06193b692b7b30, b734dc0348db42558a176fa0e13a1314; no voters:
I20260810 19:37:08.562021 20019 raft_consensus.cc:2781] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:37:08.562160 20019 raft_consensus.cc:482] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:37:08.562282 20019 raft_consensus.cc:3037] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:08.564703 20019 raft_consensus.cc:504] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.565392 20019 leader_election.cc:288] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [CANDIDATE]: Term 1 election: Requested vote from peers 93c1edb72250444c9bcef19592c06718 (127.29.233.131:42263), b734dc0348db42558a176fa0e13a1314 (127.29.233.130:39433)
I20260810 19:37:08.565814 19944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b7b4f586b4d99ab60dd896bffceac" candidate_uuid: "4c2be8f09200444baa06193b692b7b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c1edb72250444c9bcef19592c06718"
I20260810 19:37:08.565933 19799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b7b4f586b4d99ab60dd896bffceac" candidate_uuid: "4c2be8f09200444baa06193b692b7b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b734dc0348db42558a176fa0e13a1314"
I20260810 19:37:08.566059 19944 raft_consensus.cc:3037] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:08.566140 19799 raft_consensus.cc:3037] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:08.569432 19799 raft_consensus.cc:2454] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2be8f09200444baa06193b692b7b30 in term 1.
I20260810 19:37:08.569423 19944 raft_consensus.cc:2454] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2be8f09200444baa06193b692b7b30 in term 1.
I20260810 19:37:08.570022 19584 leader_election.cc:302] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2be8f09200444baa06193b692b7b30, 93c1edb72250444c9bcef19592c06718; no voters:
W20260810 19:37:08.570103 19700 tablet.cc:2369] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:08.570451 20019 raft_consensus.cc:2781] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:08.572058 20019 raft_consensus.cc:686] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 1 LEADER]: Becoming Leader. State: Replica: 4c2be8f09200444baa06193b692b7b30, State: Running, Role: LEADER
I20260810 19:37:08.572609 20019 consensus_queue.cc:237] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } }
I20260810 19:37:08.576885 19493 catalog_manager.cc:5495] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2be8f09200444baa06193b692b7b30 (127.29.233.129). New cstate: current_term: 1 leader_uuid: "4c2be8f09200444baa06193b692b7b30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2be8f09200444baa06193b692b7b30" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 40223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 } health_report { overall_health: UNKNOWN } } }
I20260810 19:37:08.654215 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.648924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.29.233.129:40223)
0810 19:37:08.649117 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.649122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.649140 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:37:08.649351 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.649355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.649462 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.649590 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:08.649948 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.650696 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.652122 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.652530 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.652636 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.653133 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:08.653655 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:08.653769 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.653927 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.654010 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 19:37:08.663482 19944 raft_consensus.cc:1261] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 1 FOLLOWER]: Refusing update from remote peer 4c2be8f09200444baa06193b692b7b30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:08.663492 19799 raft_consensus.cc:1261] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 1 FOLLOWER]: Refusing update from remote peer 4c2be8f09200444baa06193b692b7b30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:08.664610 20023 consensus_queue.cc:1035] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c1edb72250444c9bcef19592c06718" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:08.665002 20019 consensus_queue.cc:1035] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b734dc0348db42558a176fa0e13a1314" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:08.677948 20026 mvcc.cc:205] Tried to move back new op lower bound from 7317056014993842176 to 7317056014637506560. Current Snapshot: MvccSnapshot[applied={T|T < 7317056014993842176}]
I20260810 19:37:08.693331 19492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:08.693634 19492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:08.697235 19492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58a3fb8111c348b79390f2d9430d4612: Sending DeleteTablet for 3 replicas of tablet c85b7b4f586b4d99ab60dd896bffceac
I20260810 19:37:08.698608 19922 tablet_service.cc:1514] Processing DeleteTablet for tablet c85b7b4f586b4d99ab60dd896bffceac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:08 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:33798
I20260810 19:37:08.698621 19776 tablet_service.cc:1514] Processing DeleteTablet for tablet c85b7b4f586b4d99ab60dd896bffceac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:08 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:36552
I20260810 19:37:08.699002 19631 tablet_service.cc:1514] Processing DeleteTablet for tablet c85b7b4f586b4d99ab60dd896bffceac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:08 UTC) from {username='kudu', principal='kudu/127.29.233.190@KRBTEST.COM'} at 127.0.0.1:44404
I20260810 19:37:08.700677 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 19565
I20260810 19:37:08.700902 20039 tablet_replica.cc:330] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: stopping tablet replica
I20260810 19:37:08.701095 20040 tablet_replica.cc:330] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: stopping tablet replica
I20260810 19:37:08.701486 20040 raft_consensus.cc:2229] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:08.701534 20039 raft_consensus.cc:2229] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:08.702059 20040 raft_consensus.cc:2258] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:08.702071 20039 raft_consensus.cc:2258] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:08.704463 20040 ts_tablet_manager.cc:1905] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:08.704607 20039 ts_tablet_manager.cc:1905] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:08.710147 20042 tablet_replica.cc:330] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: stopping tablet replica
I20260810 19:37:08.711931 20040 ts_tablet_manager.cc:1918] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:08.712126 20040 log.cc:1196] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/c85b7b4f586b4d99ab60dd896bffceac
I20260810 19:37:08.712705 20040 ts_tablet_manager.cc:1939] T c85b7b4f586b4d99ab60dd896bffceac P 93c1edb72250444c9bcef19592c06718: Deleting consensus metadata
I20260810 19:37:08.713402 20039 ts_tablet_manager.cc:1918] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:08.713575 20039 log.cc:1196] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/c85b7b4f586b4d99ab60dd896bffceac
I20260810 19:37:08.714152 20039 ts_tablet_manager.cc:1939] T c85b7b4f586b4d99ab60dd896bffceac P b734dc0348db42558a176fa0e13a1314: Deleting consensus metadata
I20260810 19:37:08.710686 20042 raft_consensus.cc:2229] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:37:08.714833 19477 catalog_manager.cc:4841] TS 93c1edb72250444c9bcef19592c06718 (127.29.233.131:42263): tablet c85b7b4f586b4d99ab60dd896bffceac (table test-table [id=9e6c7b87c58a4257a60e7840abd660f3]) successfully deleted
I20260810 19:37:08.715046 20042 raft_consensus.cc:2258] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 10 19:37:08 dist-test-slave-tm1g krb5kdc[19441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.129@KRBTEST.COM
I20260810 19:37:08.717701 19475 catalog_manager.cc:4841] TS b734dc0348db42558a176fa0e13a1314 (127.29.233.130:39433): tablet c85b7b4f586b4d99ab60dd896bffceac (table test-table [id=9e6c7b87c58a4257a60e7840abd660f3]) successfully deleted
I20260810 19:37:08.718407 20042 ts_tablet_manager.cc:1905] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:08.724326 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.702718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.29.233.129:40223)
0810 19:37:08.702915 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.702921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.703093 (+ 172us) server_negotiation.cc:407] Connection header received
0810 19:37:08.703771 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.703777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.703863 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.704706 (+ 843us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:08.705861 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.707161 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.709046 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.709796 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.709908 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.716378 (+ 6470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:08.716389 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:08.716407 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:08.716463 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:08.720780 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.721728 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.721737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.721740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.721930 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.722469 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.722473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.722476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.722971 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:08.723351 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.723649 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.723917 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 19:37:08.725131 19693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:08.834794 20042 ts_tablet_manager.cc:1918] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:08.835106 20042 log.cc:1196] T c85b7b4f586b4d99ab60dd896bffceac P 4c2be8f09200444baa06193b692b7b30: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786390305122638-30630-0/minicluster-data/ts-0/wal/wals/c85b7b4f586b4d99ab60dd896bffceac
I20260810 19:37:08.835592 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 19565
I20260810 19:37:08.856791 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 19707
W20260810 19:37:08.856779 19476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.233.129:40223 (error 108)
I20260810 19:37:08.858991 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.858080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57536)
0810 19:37:08.858597 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.858807 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":267,"threads_started":1}
W20260810 19:37:08.859274 19476 catalog_manager.cc:4584] TS 4c2be8f09200444baa06193b692b7b30 (127.29.233.129:40223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c85b7b4f586b4d99ab60dd896bffceac: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Aug 10 19:37:08 dist-test-slave-tm1g krb5kdc[19441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.130@KRBTEST.COM
I20260810 19:37:08.873942 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.857657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58602 (local address 127.29.233.130:39433)
0810 19:37:08.857876 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.857883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.857992 (+ 109us) server_negotiation.cc:407] Connection header received
0810 19:37:08.858232 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.858237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.858354 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.858476 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:08.860124 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.860964 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.862509 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.863003 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.863139 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:08.867956 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:08.867978 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:08.867996 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:08.868060 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:08.870917 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.871757 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.871772 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.871774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.871912 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:08.872334 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:08.872339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:08.872342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:08.872729 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:08.873041 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:08.873339 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 19:37:08.873564 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:37:08.874694 19841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:08.913100 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.912630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57548)
0810 19:37:08.912851 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.912954 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 19:37:08.914361 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.913923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57550)
0810 19:37:08.914080 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.914160 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 19:37:08.967588 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 19707
I20260810 19:37:08.974982 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.974540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57558)
0810 19:37:08.974736 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.974826 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 19:37:08.976176 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.975822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57574)
0810 19:37:08.975966 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:08.976039 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 19:37:08.989378 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 19854
Aug 10 19:37:09 dist-test-slave-tm1g krb5kdc[19441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.131@KRBTEST.COM
I20260810 19:37:09.009199 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:08.990193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.29.233.131:42263)
0810 19:37:08.990410 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:08.990414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:08.990434 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:37:08.990541 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:08.990545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:08.990609 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:08.990696 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:08.992049 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.992785 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.994043 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:08.994649 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:08.994748 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:09.003841 (+ 9093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:09.003858 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:09.003875 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:09.003920 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:09.006607 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:09.007297 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:09.007303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:09.007306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:09.007454 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:09.007772 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:09.007775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:09.007777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:09.008138 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:09.008356 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:09.008653 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 19:37:09.008815 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 19:37:09.009873 19985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:09.056916 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:09.056360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57584)
0810 19:37:09.056634 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:09.056732 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260810 19:37:09.058122 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:09.057710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57586)
0810 19:37:09.057916 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:09.057987 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 19:37:09.100960 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 19854
I20260810 19:37:09.121526 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 19455
I20260810 19:37:09.122824 19527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:09.236424 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:09.235542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:40223 (local address 127.0.0.1:57598)
0810 19:37:09.235990 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:09.236139 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 19:37:09.236673 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 19455
2026-08-10T19:37:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4779 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T19:37:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:09Z Disabled control of system clock
I20260810 19:37:09.334863 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:34739
--webserver_interface=127.29.233.190
--webserver_port=0
--builtin_ntp_servers=127.29.233.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:34739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 19:37:09.689304 20052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:09.689616 20052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:09.689699 20052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:09.696986 20052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:09.697096 20052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:09.697134 20052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:09.697193 20052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:09.697243 20052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:09.706931 20052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:34739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:34739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:09.708288 20052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:09.710088 20052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:09.722165 20059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:09.722220 20062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:09.722229 20060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:09.723125 20052 server_base.cc:1034] running on GCE node
I20260810 19:37:09.723935 20052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:09.725396 20052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:09.726626 20052 hybrid_clock.cc:648] HybridClock initialized: now 1786390629726574 us; error 62 us; skew 500 ppm
I20260810 19:37:09.726965 20052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:09.729261 20052 webserver.cc:460] Webserver started at http://127.29.233.190:39667/ using document root <none> and password file <none>
I20260810 19:37:09.729960 20052 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:09.730103 20052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:09.730516 20052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:09.733395 20052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "f5a2e584f684477e844350de412b2ba7"
format_stamp: "Formatted at 2026-08-10 19:37:09 on dist-test-slave-tm1g"
server_key: "0d47e25d41016e53a9ce1db8e7d4d257"
server_key_iv: "5e25059b272f957b78f447207b78eb5d"
server_key_version: "encryptionkey@0"
I20260810 19:37:09.734220 20052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "f5a2e584f684477e844350de412b2ba7"
format_stamp: "Formatted at 2026-08-10 19:37:09 on dist-test-slave-tm1g"
server_key: "0d47e25d41016e53a9ce1db8e7d4d257"
server_key_iv: "5e25059b272f957b78f447207b78eb5d"
server_key_version: "encryptionkey@0"
I20260810 19:37:09.739444 20052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 19:37:09.744745 20068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:09.746600 20052 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 19:37:09.746788 20052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "f5a2e584f684477e844350de412b2ba7"
format_stamp: "Formatted at 2026-08-10 19:37:09 on dist-test-slave-tm1g"
server_key: "0d47e25d41016e53a9ce1db8e7d4d257"
server_key_iv: "5e25059b272f957b78f447207b78eb5d"
server_key_version: "encryptionkey@0"
I20260810 19:37:09.747035 20052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:09.760052 20052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:09.760856 20052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:09.761157 20052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:09.780076 20052 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:34739
I20260810 19:37:09.780095 20121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:34739 every 8 connection(s)
I20260810 19:37:09.781455 20052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:09.785229 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:09.786520 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20052
I20260810 19:37:09.786705 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:09.787181 30630 external_mini_cluster.cc:1428] Setting key 276dc8776b2b447983e43792cdfef87d
I20260810 19:37:09.799360 20122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7: Bootstrap starting.
I20260810 19:37:09.800838 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:09.789130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.29.233.190:34739)
0810 19:37:09.790186 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:09.790196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:09.790241 (+ 45us) server_negotiation.cc:407] Connection header received
0810 19:37:09.791848 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:09.791871 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:09.792336 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:09.792642 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:09.793245 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:09.794576 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:09.796556 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:09.797228 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:09.797349 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:09.798606 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:09.798613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:09.798624 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:09.799141 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:09.799220 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:09.799239 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:09.799321 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:09.799621 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 19:37:09.799951 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":414,"threads_started":1}
I20260810 19:37:09.802722 20122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:09.803851 20122 log.cc:824] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:09.806733 20122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7: No bootstrap required, opened a new log
I20260810 19:37:09.813917 20122 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a2e584f684477e844350de412b2ba7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34739 } }
I20260810 19:37:09.814309 20122 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:09.814474 20122 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a2e584f684477e844350de412b2ba7, State: Initialized, Role: FOLLOWER
I20260810 19:37:09.815065 20122 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a2e584f684477e844350de412b2ba7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34739 } }
I20260810 19:37:09.815310 20122 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:09.815448 20122 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:09.815629 20122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:09.817806 20122 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a2e584f684477e844350de412b2ba7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34739 } }
I20260810 19:37:09.818379 20122 leader_election.cc:302] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5a2e584f684477e844350de412b2ba7; no voters:
I20260810 19:37:09.818979 20122 leader_election.cc:288] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:09.819132 20127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:09.820535 20127 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [term 1 LEADER]: Becoming Leader. State: Replica: f5a2e584f684477e844350de412b2ba7, State: Running, Role: LEADER
I20260810 19:37:09.821029 20127 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a2e584f684477e844350de412b2ba7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34739 } }
I20260810 19:37:09.821523 20122 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:09.824168 20129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5a2e584f684477e844350de412b2ba7. Latest consensus state: current_term: 1 leader_uuid: "f5a2e584f684477e844350de412b2ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a2e584f684477e844350de412b2ba7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34739 } } }
I20260810 19:37:09.825172 20129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:09.824168 20128 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5a2e584f684477e844350de412b2ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a2e584f684477e844350de412b2ba7" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 34739 } } }
I20260810 19:37:09.825795 20128 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:09.828217 20136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:09.834085 20136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:09.843791 20136 catalog_manager.cc:1347] Generated new cluster ID: fda5927411f84b26a8c36dc918e1adb7
I20260810 19:37:09.843932 20136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:09.867515 20136 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:09.868791 20136 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:09.879174 20136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7: Generated new TSK 0
I20260810 19:37:09.880110 20136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:09.890229 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.129:0
--local_ip_for_outbound_sockets=127.29.233.129
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:34739
--builtin_ntp_servers=127.29.233.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 19:37:10.248096 20147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:10.248375 20147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:10.248476 20147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:10.255985 20147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:10.256086 20147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:10.256193 20147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.129
I20260810 19:37:10.265097 20147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.233.129
--webserver_port=0
--tserver_master_addrs=127.29.233.190:34739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.129
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:10.266388 20147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:10.268162 20147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:10.281623 20157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:10.281718 20155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:10.281679 20152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:10.282786 20147 server_base.cc:1034] running on GCE node
I20260810 19:37:10.283389 20147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:10.284358 20147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:10.285601 20147 hybrid_clock.cc:648] HybridClock initialized: now 1786390630285571 us; error 58 us; skew 500 ppm
I20260810 19:37:10.285926 20147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:10.288275 20147 webserver.cc:460] Webserver started at http://127.29.233.129:41605/ using document root <none> and password file <none>
I20260810 19:37:10.288982 20147 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:10.289117 20147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:10.289448 20147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:10.292443 20147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data/instance:
uuid: "202a3fe043f94de88c539e0ce6a326fd"
format_stamp: "Formatted at 2026-08-10 19:37:10 on dist-test-slave-tm1g"
server_key: "d8b55bcee91f128c30c0cbe834290aa2"
server_key_iv: "05fbfaa4768ec1f3e3b26480e2a5c63f"
server_key_version: "encryptionkey@0"
I20260810 19:37:10.293246 20147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance:
uuid: "202a3fe043f94de88c539e0ce6a326fd"
format_stamp: "Formatted at 2026-08-10 19:37:10 on dist-test-slave-tm1g"
server_key: "d8b55bcee91f128c30c0cbe834290aa2"
server_key_iv: "05fbfaa4768ec1f3e3b26480e2a5c63f"
server_key_version: "encryptionkey@0"
I20260810 19:37:10.298374 20147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 19:37:10.301977 20163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:10.303375 20147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 19:37:10.303550 20147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/wal
uuid: "202a3fe043f94de88c539e0ce6a326fd"
format_stamp: "Formatted at 2026-08-10 19:37:10 on dist-test-slave-tm1g"
server_key: "d8b55bcee91f128c30c0cbe834290aa2"
server_key_iv: "05fbfaa4768ec1f3e3b26480e2a5c63f"
server_key_version: "encryptionkey@0"
I20260810 19:37:10.303761 20147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:10.315510 20147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:10.316270 20147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:10.316574 20147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:10.317639 20147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:10.319353 20147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:10.319455 20147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:10.319612 20147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:10.319708 20147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:10.369907 20147 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.129:45013
I20260810 19:37:10.369973 20282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.129:45013 every 8 connection(s)
I20260810 19:37:10.372200 20147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/data/info.pb
I20260810 19:37:10.373862 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20147
I20260810 19:37:10.374172 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-0/wal/instance
I20260810 19:37:10.375197 30630 external_mini_cluster.cc:1428] Setting key f29f71e4c33538a61aeae1c21e032088
I20260810 19:37:10.380904 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.130:0
--local_ip_for_outbound_sockets=127.29.233.130
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:34739
--builtin_ntp_servers=127.29.233.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 19:37:10.398352 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:10.376941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:47807 (local address 127.29.233.190:34739)
0810 19:37:10.377745 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:10.377755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:10.380921 (+ 3166us) server_negotiation.cc:407] Connection header received
0810 19:37:10.386733 (+ 5812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:10.386754 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:10.387000 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:10.387200 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:10.389107 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.390449 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.393190 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.393935 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.394201 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:10.396634 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:10.396648 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:10.396653 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:10.396675 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:10.396739 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:10.396761 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:10.396962 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:10.397342 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 19:37:10.397555 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":351,"threads_started":1}
I20260810 19:37:10.400264 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:10.378255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:34739 (local address 127.29.233.129:47807)
0810 19:37:10.380478 (+ 2223us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:10.380581 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:10.386135 (+ 5554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:10.387625 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:10.387645 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:10.387809 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:10.388818 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:10.388849 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.390690 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.390713 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:10.392952 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:10.392967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.394409 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.394418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:10.394839 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:10.396057 (+ 1218us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:37:10.396060 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:10.396138 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:37:10.396269 (+ 131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:10.396940 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:10.396966 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:10.396988 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 19:37:10.397332 (+ 344us) client_negotiation.cc:240] Negotiation successful
0810 19:37:10.398149 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"thread_start_us":681,"threads_started":1}
I20260810 19:37:10.403097 20283 heartbeater.cc:347] Connected to a master server at 127.29.233.190:34739
I20260810 19:37:10.403661 20283 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:10.404924 20283 heartbeater.cc:511] Master 127.29.233.190:34739 requested a full tablet report, sending...
I20260810 19:37:10.409044 20086 ts_manager.cc:194] Registered new tserver with Master: 202a3fe043f94de88c539e0ce6a326fd (127.29.233.129:45013)
I20260810 19:37:10.412945 20086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.129:47807
W20260810 19:37:10.798995 20287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:10.799278 20287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:10.799365 20287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:10.806677 20287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:10.806777 20287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:10.806887 20287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.130
I20260810 19:37:10.816169 20287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.233.130
--webserver_port=0
--tserver_master_addrs=127.29.233.190:34739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.130
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:10.817490 20287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:10.819386 20287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:10.832948 20295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:10.833086 20297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:10.833511 20293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:10.833813 20287 server_base.cc:1034] running on GCE node
I20260810 19:37:10.834535 20287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:10.835572 20287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:10.836871 20287 hybrid_clock.cc:648] HybridClock initialized: now 1786390630836801 us; error 57 us; skew 500 ppm
I20260810 19:37:10.837234 20287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:10.839807 20287 webserver.cc:460] Webserver started at http://127.29.233.130:45285/ using document root <none> and password file <none>
I20260810 19:37:10.840579 20287 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:10.840698 20287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:10.841043 20287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:10.844221 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data/instance:
uuid: "235bf1cbd7ec4af8a207e88c563049e7"
format_stamp: "Formatted at 2026-08-10 19:37:10 on dist-test-slave-tm1g"
server_key: "b480e251a8191d68140a7c43d9952752"
server_key_iv: "b2a7dac433e3c077bc7e3e863776bdc5"
server_key_version: "encryptionkey@0"
I20260810 19:37:10.845067 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance:
uuid: "235bf1cbd7ec4af8a207e88c563049e7"
format_stamp: "Formatted at 2026-08-10 19:37:10 on dist-test-slave-tm1g"
server_key: "b480e251a8191d68140a7c43d9952752"
server_key_iv: "b2a7dac433e3c077bc7e3e863776bdc5"
server_key_version: "encryptionkey@0"
I20260810 19:37:10.850690 20287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 19:37:10.854449 20304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:10.855789 20287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:10.855979 20287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal
uuid: "235bf1cbd7ec4af8a207e88c563049e7"
format_stamp: "Formatted at 2026-08-10 19:37:10 on dist-test-slave-tm1g"
server_key: "b480e251a8191d68140a7c43d9952752"
server_key_iv: "b2a7dac433e3c077bc7e3e863776bdc5"
server_key_version: "encryptionkey@0"
I20260810 19:37:10.856209 20287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:10.872545 20287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:10.873401 20287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:10.873729 20287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:10.874773 20287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:10.876287 20287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:10.876443 20287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:10.876574 20287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:10.876667 20287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:10.911221 20287 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.130:39673
I20260810 19:37:10.911283 20429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.130:39673 every 8 connection(s)
I20260810 19:37:10.912577 20287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/data/info.pb
I20260810 19:37:10.918231 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20287
I20260810 19:37:10.918434 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal/instance
I20260810 19:37:10.918985 30630 external_mini_cluster.cc:1428] Setting key 9eaac87b823337423e205669f3bf0d78
I20260810 19:37:10.922205 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.233.131:0
--local_ip_for_outbound_sockets=127.29.233.131
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:34739
--builtin_ntp_servers=127.29.233.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 19:37:10.927486 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:10.915383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.130:52905 (local address 127.29.233.190:34739)
0810 19:37:10.915978 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:10.915982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:10.916981 (+ 999us) server_negotiation.cc:407] Connection header received
0810 19:37:10.919352 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:10.919356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:10.919432 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:10.919534 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:10.920730 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.921694 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.924419 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.924820 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.924927 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:10.926612 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:10.926617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:10.926620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:10.926639 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:10.926678 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:10.926689 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:10.926761 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:10.927025 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 19:37:10.927151 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":385,"threads_started":1}
I20260810 19:37:10.928854 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:10.915827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:34739 (local address 127.29.233.130:52905)
0810 19:37:10.916816 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:10.916860 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:10.919058 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:10.919773 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:10.919781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:10.919918 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:10.920561 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:10.920578 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.921878 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.921886 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:10.924241 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:10.924262 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:10.925051 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:10.925060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:10.925405 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:10.926221 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:37:10.926227 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:10.926283 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:37:10.926415 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:10.926789 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:10.926802 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:10.926812 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 19:37:10.926990 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 19:37:10.927418 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":300,"threads_started":1}
I20260810 19:37:10.930397 20430 heartbeater.cc:347] Connected to a master server at 127.29.233.190:34739
I20260810 19:37:10.930773 20430 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:10.931566 20430 heartbeater.cc:511] Master 127.29.233.190:34739 requested a full tablet report, sending...
I20260810 19:37:10.933956 20086 ts_manager.cc:194] Registered new tserver with Master: 235bf1cbd7ec4af8a207e88c563049e7 (127.29.233.130:39673)
I20260810 19:37:10.934881 20086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.130:52905
W20260810 19:37:11.268731 20435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:11.269012 20435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:11.269168 20435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:11.276623 20435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:11.276726 20435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:11.276932 20435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.233.131
I20260810 19:37:11.286290 20435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.233.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.233.131
--webserver_port=0
--tserver_master_addrs=127.29.233.190:34739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.233.131
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:11.287691 20435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:11.289522 20435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:11.302733 20440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:11.302733 20443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:11.302793 20441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:11.304394 20435 server_base.cc:1034] running on GCE node
I20260810 19:37:11.305024 20435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:11.306046 20435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:11.307284 20435 hybrid_clock.cc:648] HybridClock initialized: now 1786390631307216 us; error 46 us; skew 500 ppm
I20260810 19:37:11.307684 20435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 19:37:11.309980 20435 webserver.cc:460] Webserver started at http://127.29.233.131:34287/ using document root <none> and password file <none>
I20260810 19:37:11.310907 20435 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:11.311048 20435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:11.311378 20435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:11.314250 20435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data/instance:
uuid: "5e766ccc1e2e4c6ca17d91d1c1614303"
format_stamp: "Formatted at 2026-08-10 19:37:11 on dist-test-slave-tm1g"
server_key: "6401156763053e2a3339c5b4cf0a3b7c"
server_key_iv: "e6af79d2d5e27a9c5131cb0813866192"
server_key_version: "encryptionkey@0"
I20260810 19:37:11.315135 20435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance:
uuid: "5e766ccc1e2e4c6ca17d91d1c1614303"
format_stamp: "Formatted at 2026-08-10 19:37:11 on dist-test-slave-tm1g"
server_key: "6401156763053e2a3339c5b4cf0a3b7c"
server_key_iv: "e6af79d2d5e27a9c5131cb0813866192"
server_key_version: "encryptionkey@0"
I20260810 19:37:11.320407 20435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:37:11.324033 20450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:11.325350 20435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:37:11.325551 20435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal
uuid: "5e766ccc1e2e4c6ca17d91d1c1614303"
format_stamp: "Formatted at 2026-08-10 19:37:11 on dist-test-slave-tm1g"
server_key: "6401156763053e2a3339c5b4cf0a3b7c"
server_key_iv: "e6af79d2d5e27a9c5131cb0813866192"
server_key_version: "encryptionkey@0"
I20260810 19:37:11.325759 20435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:11.338820 20435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:11.339592 20435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:11.339887 20435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:11.340824 20435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 19:37:11.342218 20435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 19:37:11.342381 20435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:11.342515 20435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 19:37:11.342605 20435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:11.375810 20435 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.131:38549
I20260810 19:37:11.375852 20571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.131:38549 every 8 connection(s)
I20260810 19:37:11.377043 20435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/data/info.pb
I20260810 19:37:11.387245 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20435
I20260810 19:37:11.387441 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal/instance
I20260810 19:37:11.388110 30630 external_mini_cluster.cc:1428] Setting key 4e2b3f4d492f14001913ef9ee5201156
I20260810 19:37:11.391623 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.379644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.131:54091 (local address 127.29.233.190:34739)
0810 19:37:11.379880 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.379886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.381150 (+ 1264us) server_negotiation.cc:407] Connection header received
0810 19:37:11.383025 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.383030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.383103 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.383198 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:11.384260 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.385042 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.387167 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.387867 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.388031 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.390842 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:11.390849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:11.390855 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:11.390876 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:11.390917 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:11.390935 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:11.391036 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.391299 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.391383 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 19:37:11.392612 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.380104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:34739 (local address 127.29.233.131:54091)
0810 19:37:11.380986 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.381028 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:11.382751 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:11.383450 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:11.383459 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:11.383540 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:11.384110 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.384126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.385179 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.385183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.386931 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.386946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.389366 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.389377 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.389722 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.390529 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 19:37:11.390532 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:11.390571 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 19:37:11.390645 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:11.391051 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:11.391060 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:11.391066 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 19:37:11.391250 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 19:37:11.391563 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":270,"threads_started":1}
I20260810 19:37:11.393911 20572 heartbeater.cc:347] Connected to a master server at 127.29.233.190:34739
I20260810 19:37:11.394215 20572 heartbeater.cc:464] Registering TS with master...
I20260810 19:37:11.394977 20572 heartbeater.cc:511] Master 127.29.233.190:34739 requested a full tablet report, sending...
I20260810 19:37:11.396322 20086 ts_manager.cc:194] Registered new tserver with Master: 5e766ccc1e2e4c6ca17d91d1c1614303 (127.29.233.131:38549)
I20260810 19:37:11.397184 20086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.233.131:54091
I20260810 19:37:11.403461 30630 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 19:37:11.413885 20433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.407866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.29.233.190:34739)
0810 19:37:11.408075 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.408081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.408510 (+ 429us) server_negotiation.cc:407] Connection header received
0810 19:37:11.408724 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.408730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.408815 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.408923 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:11.409410 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.410152 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.411531 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.411968 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.412120 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.413129 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:11.413141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:11.413145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:11.413162 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:11.413195 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:11.413206 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:11.413307 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.413453 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.413548 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:37:11.418485 20283 heartbeater.cc:503] Master 127.29.233.190:34739 was elected leader, sending a full tablet report...
I20260810 19:37:11.417675 20086 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 19:37:11.425449 20086 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 19:37:11.458810 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.438377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.29.233.131:38549)
0810 19:37:11.450241 (+ 11864us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.450254 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.450329 (+ 75us) server_negotiation.cc:407] Connection header received
0810 19:37:11.450456 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.450468 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.450873 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.451094 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:11.452399 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.453596 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.456360 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.457562 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.457787 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.458034 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.458236 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.458444 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11629,"thread_start_us":11390,"threads_started":1}
I20260810 19:37:11.458814 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.438035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:38549 (local address 127.0.0.1:46122)
0810 19:37:11.445175 (+ 7140us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.445218 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:11.445431 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:11.451434 (+ 6003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:11.451444 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:11.451480 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:11.452172 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.452188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.453727 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.453735 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.456131 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.456153 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.457764 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.457772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.457964 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.458027 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 19:37:11.458183 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 19:37:11.458399 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6886,"mutex_wait_us":70,"thread_start_us":6703,"threads_started":1}
I20260810 19:37:11.459452 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.438294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:39673 (local address 127.0.0.1:39182)
0810 19:37:11.447722 (+ 9428us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.447745 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:11.447883 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:11.451724 (+ 3841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:11.451734 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:11.451763 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:11.452351 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.452367 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.454435 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.454440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.456715 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.456726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.458614 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.458624 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.458804 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.458836 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 19:37:11.458900 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 19:37:11.459030 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9249,"mutex_wait_us":152,"thread_start_us":11755,"threads_started":1}
I20260810 19:37:11.459919 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.445663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45204)
0810 19:37:11.449291 (+ 3628us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.449324 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:11.449451 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:11.451471 (+ 2020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:11.451476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:11.451503 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:11.452628 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.452642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.454498 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.454505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.456895 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.456909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.459081 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.459087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.459260 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.459297 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 19:37:11.459396 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 19:37:11.459535 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3481,"mutex_wait_us":617,"thread_start_us":2314,"threads_started":1}
I20260810 19:37:11.460707 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.438958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39182 (local address 127.29.233.130:39673)
0810 19:37:11.451066 (+ 12108us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.451074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.451100 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:37:11.451211 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.451221 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.451494 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.452486 (+ 992us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:11.452685 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.454259 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.456903 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.458381 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.459834 (+ 1453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.460069 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.460138 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.460295 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11959,"thread_start_us":11835,"threads_started":1}
I20260810 19:37:11.461812 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.446265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45204 (local address 127.29.233.129:45013)
0810 19:37:11.450767 (+ 4502us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.450780 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.450813 (+ 33us) server_negotiation.cc:407] Connection header received
0810 19:37:11.450917 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.450923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.451185 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.451900 (+ 715us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:11.452840 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.454381 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.457093 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.458560 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.458745 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.458957 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.460371 (+ 1414us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.460549 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4317,"thread_start_us":4159,"threads_started":1}
I20260810 19:37:11.463806 20502 tablet_service.cc:1467] Processing CreateTablet for tablet d4bcfb4333544b7988831d5074d1691c (DEFAULT_TABLE table=test-table [id=8ef564b7392a4c8a9192befba5fc0698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:11.464198 20355 tablet_service.cc:1467] Processing CreateTablet for tablet d4bcfb4333544b7988831d5074d1691c (DEFAULT_TABLE table=test-table [id=8ef564b7392a4c8a9192befba5fc0698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:11.465591 20502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4bcfb4333544b7988831d5074d1691c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:11.465227 20213 tablet_service.cc:1467] Processing CreateTablet for tablet d4bcfb4333544b7988831d5074d1691c (DEFAULT_TABLE table=test-table [id=8ef564b7392a4c8a9192befba5fc0698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 19:37:11.466463 20355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4bcfb4333544b7988831d5074d1691c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:11.466657 20213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4bcfb4333544b7988831d5074d1691c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:11.477314 20591 tablet_bootstrap.cc:492] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Bootstrap starting.
I20260810 19:37:11.480832 20592 tablet_bootstrap.cc:492] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: Bootstrap starting.
I20260810 19:37:11.481621 20591 tablet_bootstrap.cc:654] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:11.482445 20593 tablet_bootstrap.cc:492] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Bootstrap starting.
I20260810 19:37:11.483498 20591 log.cc:824] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:11.484584 20592 tablet_bootstrap.cc:654] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:11.485656 20593 tablet_bootstrap.cc:654] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:11.485819 20592 log.cc:824] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:11.486898 20593 log.cc:824] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:11.486970 20591 tablet_bootstrap.cc:492] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: No bootstrap required, opened a new log
I20260810 19:37:11.487267 20591 ts_tablet_manager.cc:1397] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 19:37:11.489275 20592 tablet_bootstrap.cc:492] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: No bootstrap required, opened a new log
I20260810 19:37:11.489604 20592 ts_tablet_manager.cc:1397] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 19:37:11.494673 20591 raft_consensus.cc:348] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.495185 20591 raft_consensus.cc:374] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:11.495306 20591 raft_consensus.cc:729] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e766ccc1e2e4c6ca17d91d1c1614303, State: Initialized, Role: FOLLOWER
I20260810 19:37:11.495970 20591 consensus_queue.cc:260] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.497149 20593 tablet_bootstrap.cc:492] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: No bootstrap required, opened a new log
I20260810 19:37:11.497432 20593 ts_tablet_manager.cc:1397] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260810 19:37:11.497772 20572 heartbeater.cc:503] Master 127.29.233.190:34739 was elected leader, sending a full tablet report...
I20260810 19:37:11.498389 20591 ts_tablet_manager.cc:1428] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260810 19:37:11.501260 20592 raft_consensus.cc:348] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.501715 20592 raft_consensus.cc:374] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:11.501816 20592 raft_consensus.cc:729] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 202a3fe043f94de88c539e0ce6a326fd, State: Initialized, Role: FOLLOWER
I20260810 19:37:11.502468 20592 consensus_queue.cc:260] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.504141 20592 ts_tablet_manager.cc:1428] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: Time spent starting tablet: real 0.014s user 0.008s sys 0.008s
I20260810 19:37:11.507242 20593 raft_consensus.cc:348] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.507613 20593 raft_consensus.cc:374] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:11.507730 20593 raft_consensus.cc:729] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235bf1cbd7ec4af8a207e88c563049e7, State: Initialized, Role: FOLLOWER
I20260810 19:37:11.508276 20593 consensus_queue.cc:260] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.509282 20430 heartbeater.cc:503] Master 127.29.233.190:34739 was elected leader, sending a full tablet report...
I20260810 19:37:11.509883 20593 ts_tablet_manager.cc:1428] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Time spent starting tablet: real 0.012s user 0.013s sys 0.001s
I20260810 19:37:11.564152 20599 raft_consensus.cc:482] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 19:37:11.564468 20599 raft_consensus.cc:504] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.566286 20599 leader_election.cc:288] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e766ccc1e2e4c6ca17d91d1c1614303 (127.29.233.131:38549), 235bf1cbd7ec4af8a207e88c563049e7 (127.29.233.130:39673)
I20260810 19:37:11.574411 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.566919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:37077 (local address 127.29.233.130:39673)
0810 19:37:11.567390 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.567399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.568056 (+ 657us) server_negotiation.cc:407] Connection header received
0810 19:37:11.568238 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.568243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.568323 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.568393 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:11.569037 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.570026 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.572696 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.573445 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.573620 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.573694 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.573987 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.574114 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315}
I20260810 19:37:11.574445 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.566818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.233.129:43611 (local address 127.29.233.131:38549)
0810 19:37:11.567195 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.567204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.567653 (+ 449us) server_negotiation.cc:407] Connection header received
0810 19:37:11.567905 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.567910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.567971 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.568086 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 19:37:11.568720 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.569804 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.572708 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.573497 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.573678 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.573748 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.574044 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.574172 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260810 19:37:11.574542 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.566579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.131:38549 (local address 127.29.233.129:43611)
0810 19:37:11.567532 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.567549 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:11.567743 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:11.568130 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:11.568137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:11.568168 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:11.568561 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.568574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.569935 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.569939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.572429 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.572439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.573706 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.573713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.573871 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.573894 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 19:37:11.574015 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 19:37:11.574139 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":35,"thread_start_us":606,"threads_started":1}
I20260810 19:37:11.574774 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.566849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.130:39673 (local address 127.29.233.129:37077)
0810 19:37:11.567935 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.567953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:11.568097 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:11.568454 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:11.568458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:11.568499 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 19:37:11.568867 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.568876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.570156 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.570161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.572411 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:11.572432 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.573655 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.573661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:11.573823 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.573847 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 19:37:11.573958 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 19:37:11.574100 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":81,"thread_start_us":753,"threads_started":1}
I20260810 19:37:11.575767 20526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4bcfb4333544b7988831d5074d1691c" candidate_uuid: "202a3fe043f94de88c539e0ce6a326fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" is_pre_election: true
I20260810 19:37:11.575856 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4bcfb4333544b7988831d5074d1691c" candidate_uuid: "202a3fe043f94de88c539e0ce6a326fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235bf1cbd7ec4af8a207e88c563049e7" is_pre_election: true
I20260810 19:37:11.576416 20526 raft_consensus.cc:2454] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 202a3fe043f94de88c539e0ce6a326fd in term 0.
I20260810 19:37:11.576416 20380 raft_consensus.cc:2454] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 202a3fe043f94de88c539e0ce6a326fd in term 0.
I20260810 19:37:11.577176 20166 leader_election.cc:302] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 202a3fe043f94de88c539e0ce6a326fd, 235bf1cbd7ec4af8a207e88c563049e7, 5e766ccc1e2e4c6ca17d91d1c1614303; no voters:
I20260810 19:37:11.577569 20599 raft_consensus.cc:2781] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 19:37:11.577741 20599 raft_consensus.cc:482] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 19:37:11.577872 20599 raft_consensus.cc:3037] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:11.580178 20599 raft_consensus.cc:504] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.580855 20599 leader_election.cc:288] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [CANDIDATE]: Term 1 election: Requested vote from peers 5e766ccc1e2e4c6ca17d91d1c1614303 (127.29.233.131:38549), 235bf1cbd7ec4af8a207e88c563049e7 (127.29.233.130:39673)
I20260810 19:37:11.581246 20526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4bcfb4333544b7988831d5074d1691c" candidate_uuid: "202a3fe043f94de88c539e0ce6a326fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303"
I20260810 19:37:11.581359 20380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4bcfb4333544b7988831d5074d1691c" candidate_uuid: "202a3fe043f94de88c539e0ce6a326fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235bf1cbd7ec4af8a207e88c563049e7"
I20260810 19:37:11.581450 20526 raft_consensus.cc:3037] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:11.581600 20380 raft_consensus.cc:3037] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:11.583786 20526 raft_consensus.cc:2454] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 202a3fe043f94de88c539e0ce6a326fd in term 1.
I20260810 19:37:11.584048 20380 raft_consensus.cc:2454] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 202a3fe043f94de88c539e0ce6a326fd in term 1.
I20260810 19:37:11.584203 20165 leader_election.cc:302] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 202a3fe043f94de88c539e0ce6a326fd, 5e766ccc1e2e4c6ca17d91d1c1614303; no voters:
I20260810 19:37:11.584498 20599 raft_consensus.cc:2781] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:11.584923 20599 raft_consensus.cc:686] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 1 LEADER]: Becoming Leader. State: Replica: 202a3fe043f94de88c539e0ce6a326fd, State: Running, Role: LEADER
I20260810 19:37:11.585428 20599 consensus_queue.cc:237] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } }
I20260810 19:37:11.589823 20086 catalog_manager.cc:5495] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 202a3fe043f94de88c539e0ce6a326fd (127.29.233.129). New cstate: current_term: 1 leader_uuid: "202a3fe043f94de88c539e0ce6a326fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "202a3fe043f94de88c539e0ce6a326fd" member_type: VOTER last_known_addr { host: "127.29.233.129" port: 45013 } health_report { overall_health: HEALTHY } } }
I20260810 19:37:11.613636 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.608562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45206 (local address 127.29.233.129:45013)
0810 19:37:11.608740 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.608744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.608831 (+ 87us) server_negotiation.cc:407] Connection header received
0810 19:37:11.609013 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.609017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.609123 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.609257 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:11.609571 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.610285 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.611726 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.612139 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.612229 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.612644 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:11.613179 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:11.613294 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.613368 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.613432 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 19:37:11.621901 20526 raft_consensus.cc:1261] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 1 FOLLOWER]: Refusing update from remote peer 202a3fe043f94de88c539e0ce6a326fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:11.621870 20380 raft_consensus.cc:1261] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 1 FOLLOWER]: Refusing update from remote peer 202a3fe043f94de88c539e0ce6a326fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:11.623179 20603 consensus_queue.cc:1035] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "235bf1cbd7ec4af8a207e88c563049e7" member_type: VOTER last_known_addr { host: "127.29.233.130" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:11.623590 20599 consensus_queue.cc:1035] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e766ccc1e2e4c6ca17d91d1c1614303" member_type: VOTER last_known_addr { host: "127.29.233.131" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 19:37:11.625675 20284 tablet.cc:2369] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 19:37:11.628731 20573 tablet.cc:2369] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:11.635905 20607 mvcc.cc:205] Tried to move back new op lower bound from 7317056027113398272 to 7317056026977722368. Current Snapshot: MvccSnapshot[applied={T|T < 7317056027113398272}]
I20260810 19:37:11.636747 20608 mvcc.cc:205] Tried to move back new op lower bound from 7317056027113398272 to 7317056026977722368. Current Snapshot: MvccSnapshot[applied={T|T < 7317056027113398272}]
I20260810 19:37:11.650789 20085 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:11.651090 20085 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 19:37:11.655076 20085 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5a2e584f684477e844350de412b2ba7: Sending DeleteTablet for 3 replicas of tablet d4bcfb4333544b7988831d5074d1691c
I20260810 19:37:11.656247 20355 tablet_service.cc:1514] Processing DeleteTablet for tablet d4bcfb4333544b7988831d5074d1691c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:11 UTC) from {username='slave'} at 127.0.0.1:39182
I20260810 19:37:11.656260 20213 tablet_service.cc:1514] Processing DeleteTablet for tablet d4bcfb4333544b7988831d5074d1691c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:11 UTC) from {username='slave'} at 127.0.0.1:45204
I20260810 19:37:11.657055 20502 tablet_service.cc:1514] Processing DeleteTablet for tablet d4bcfb4333544b7988831d5074d1691c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 19:37:11 UTC) from {username='slave'} at 127.0.0.1:46122
I20260810 19:37:11.659065 20618 tablet_replica.cc:330] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: stopping tablet replica
I20260810 19:37:11.659570 20618 raft_consensus.cc:2229] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:11.660187 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20147
I20260810 19:37:11.660216 20618 raft_consensus.cc:2258] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:11.662693 20618 ts_tablet_manager.cc:1905] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:11.670122 20618 ts_tablet_manager.cc:1918] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:11.670418 20618 log.cc:1196] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-2/wal/wals/d4bcfb4333544b7988831d5074d1691c
I20260810 19:37:11.671113 20618 ts_tablet_manager.cc:1939] T d4bcfb4333544b7988831d5074d1691c P 5e766ccc1e2e4c6ca17d91d1c1614303: Deleting consensus metadata
I20260810 19:37:11.673255 20070 catalog_manager.cc:4841] TS 5e766ccc1e2e4c6ca17d91d1c1614303 (127.29.233.131:38549): tablet d4bcfb4333544b7988831d5074d1691c (table test-table [id=8ef564b7392a4c8a9192befba5fc0698]) successfully deleted
I20260810 19:37:11.678965 20616 tablet_replica.cc:330] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd: stopping tablet replica
I20260810 19:37:11.679481 20616 raft_consensus.cc:2229] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 1 LEADER]: Raft consensus shutting down.
I20260810 19:37:11.679608 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.662653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45222 (local address 127.29.233.129:45013)
0810 19:37:11.662900 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.662909 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.668321 (+ 5412us) server_negotiation.cc:407] Connection header received
0810 19:37:11.668558 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.668567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.668695 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.668813 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:11.669889 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.671659 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.674201 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.675248 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.675401 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.677426 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:11.677441 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:11.677468 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:11.678530 (+ 1062us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:11.678612 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:11.678647 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:11.678739 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.679065 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.679179 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:37:11.680285 20617 tablet_replica.cc:330] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: stopping tablet replica
I20260810 19:37:11.680682 20617 raft_consensus.cc:2229] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 19:37:11.680907 20278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:11.681313 20617 raft_consensus.cc:2258] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:11.684098 20617 ts_tablet_manager.cc:1905] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 19:37:11.689574 20617 ts_tablet_manager.cc:1918] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 19:37:11.689771 20617 log.cc:1196] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Deleting WAL directory at /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786390305122638-30630-0/minicluster-data/ts-1/wal/wals/d4bcfb4333544b7988831d5074d1691c
I20260810 19:37:11.690251 20617 ts_tablet_manager.cc:1939] T d4bcfb4333544b7988831d5074d1691c P 235bf1cbd7ec4af8a207e88c563049e7: Deleting consensus metadata
I20260810 19:37:11.691970 20071 catalog_manager.cc:4841] TS 235bf1cbd7ec4af8a207e88c563049e7 (127.29.233.130:39673): tablet d4bcfb4333544b7988831d5074d1691c (table test-table [id=8ef564b7392a4c8a9192befba5fc0698]) successfully deleted
I20260810 19:37:11.780709 20616 raft_consensus.cc:2258] T d4bcfb4333544b7988831d5074d1691c P 202a3fe043f94de88c539e0ce6a326fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 19:37:11.782462 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20147
W20260810 19:37:11.800758 20070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.233.129:45013 (error 108)
I20260810 19:37:11.801025 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20287
I20260810 19:37:11.802886 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.802392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45236)
0810 19:37:11.802561 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.802694 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 19:37:11.803239 20070 catalog_manager.cc:4584] TS 202a3fe043f94de88c539e0ce6a326fd (127.29.233.129:45013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4bcfb4333544b7988831d5074d1691c: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
I20260810 19:37:11.808182 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.801873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.29.233.130:39673)
0810 19:37:11.802053 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.802057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.802075 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:37:11.802318 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.802324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.802443 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.802565 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:11.803129 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.803999 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.805191 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.805607 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.805700 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.806855 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:11.806864 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:11.806877 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:11.807455 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:11.807502 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:11.807521 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:11.807613 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.807722 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.807805 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:37:11.808924 20425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:11.857260 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.856562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45246)
0810 19:37:11.856881 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.857023 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 19:37:11.858551 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.858039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45252)
0810 19:37:11.858224 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.858353 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 19:37:11.908201 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20287
I20260810 19:37:11.919510 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.918793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45260)
0810 19:37:11.919077 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.919206 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 19:37:11.920892 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.920391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45276)
0810 19:37:11.920590 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:11.920700 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 19:37:11.933274 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20435
I20260810 19:37:11.941251 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:11.934228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46136 (local address 127.29.233.131:38549)
0810 19:37:11.934688 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:11.934696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:11.934720 (+ 24us) server_negotiation.cc:407] Connection header received
0810 19:37:11.934798 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:11.934805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:11.934953 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:11.935075 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:11.935532 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.936829 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.938148 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:11.938598 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:11.938697 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:11.939768 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:11.939777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:11.939792 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:11.940413 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:11.940461 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:11.940487 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:11.940586 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:11.940763 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 19:37:11.940854 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260810 19:37:11.941973 20567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:12.001904 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:12.001280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45284)
0810 19:37:12.001574 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:12.001700 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260810 19:37:12.003180 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:12.002733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.129:45013 (local address 127.0.0.1:45296)
0810 19:37:12.002959 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:12.003049 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260810 19:37:12.032687 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20435
I20260810 19:37:12.053411 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20052
I20260810 19:37:12.054594 20117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:12.166860 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20052
2026-08-10T19:37:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2934 ms)
[----------] 6 tests from EncryptionPolicyTest (23108 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:12 dist-test-slave-tm1g krb5kdc[20627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:12 dist-test-slave-tm1g krb5kdc[20627](info): set up 2 sockets
Aug 10 19:37:12 dist-test-slave-tm1g krb5kdc[20627](info): commencing operation
krb5kdc: starting...
W20260810 19:37:13.370435 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:13 dist-test-slave-tm1g krb5kdc[20627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:13.698232 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45439
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:14.049150 20640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:14.049511 20640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:14.049672 20640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:14.056902 20640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:14.057060 20640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:14.057163 20640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:14.057267 20640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:14.057343 20640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:14.067230 20640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45439
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:14.068567 20640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:14.069903 20640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:14.082455 20645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:14.082453 20646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:14.082409 20648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:14.083230 20640 server_base.cc:1034] running on GCE node
I20260810 19:37:14.084043 20640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:14.085678 20640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:14.086925 20640 hybrid_clock.cc:648] HybridClock initialized: now 1786390634086869 us; error 61 us; skew 500 ppm
I20260810 19:37:14.087325 20640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:14 dist-test-slave-tm1g krb5kdc[20627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:14.099740 20640 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:14.101348 20640 webserver.cc:460] Webserver started at http://127.0.0.1:39569/ using document root <none> and password file <none>
I20260810 19:37:14.102083 20640 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:14.102238 20640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:14.102591 20640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:14.105700 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "9260a93630614775965df8b88ab92d7f"
format_stamp: "Formatted at 2026-08-10 19:37:14 on dist-test-slave-tm1g"
server_key: "77dc6e3efca986d2d1196f570f22af38"
server_key_iv: "9aaa46107aa0c59384b6c6c78839bd54"
server_key_version: "encryptionkey@0"
I20260810 19:37:14.106570 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "9260a93630614775965df8b88ab92d7f"
format_stamp: "Formatted at 2026-08-10 19:37:14 on dist-test-slave-tm1g"
server_key: "77dc6e3efca986d2d1196f570f22af38"
server_key_iv: "9aaa46107aa0c59384b6c6c78839bd54"
server_key_version: "encryptionkey@0"
I20260810 19:37:14.111768 20640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:37:14.115237 20657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:14.116680 20640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:37:14.116902 20640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "9260a93630614775965df8b88ab92d7f"
format_stamp: "Formatted at 2026-08-10 19:37:14 on dist-test-slave-tm1g"
server_key: "77dc6e3efca986d2d1196f570f22af38"
server_key_iv: "9aaa46107aa0c59384b6c6c78839bd54"
server_key_version: "encryptionkey@0"
I20260810 19:37:14.117115 20640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:14.132593 20640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:14.136950 20640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:14.137241 20640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:14.156095 20640 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45439
I20260810 19:37:14.156129 20711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45439 every 8 connection(s)
I20260810 19:37:14.157474 20640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:14.161535 20712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:14.162109 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20640
I20260810 19:37:14.162395 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:14.163122 30630 external_mini_cluster.cc:1428] Setting key 5df64414d683acf8fb33457d25088512
I20260810 19:37:14.175968 20712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f: Bootstrap starting.
Aug 10 19:37:14 dist-test-slave-tm1g krb5kdc[20627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 19:37:14.179806 20712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:14.180905 20712 log.cc:824] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:14.183962 20712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f: No bootstrap required, opened a new log
I20260810 19:37:14.187570 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:14.165192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58111 (local address 127.0.0.1:45439)
0810 19:37:14.166071 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:14.166089 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:14.166135 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:37:14.167950 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:14.167974 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:14.168383 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:14.168706 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:14.170634 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:14.171986 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:14.173896 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:14.174625 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:14.174765 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:14.179820 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:14.179828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:14.179841 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:14.179879 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:14.183303 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:14.184138 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:14.184144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:14.184150 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:14.184309 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:14.184825 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:14.184835 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:14.184837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:14.185382 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:14.185759 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:14.186226 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 19:37:14.186582 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":428,"threads_started":1}
I20260810 19:37:14.191743 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:14.192735 20712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260a93630614775965df8b88ab92d7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45439 } }
I20260810 19:37:14.193146 20712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:14.193248 20712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9260a93630614775965df8b88ab92d7f, State: Initialized, Role: FOLLOWER
I20260810 19:37:14.193867 20712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260a93630614775965df8b88ab92d7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45439 } }
I20260810 19:37:14.194144 20712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:14.194295 20712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:14.194527 20712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:14.198009 20712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260a93630614775965df8b88ab92d7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45439 } }
I20260810 19:37:14.198679 20712 leader_election.cc:302] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9260a93630614775965df8b88ab92d7f; no voters:
I20260810 19:37:14.199414 20712 leader_election.cc:288] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:14.199510 20723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:14.202073 20712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:14.206219 20723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [term 1 LEADER]: Becoming Leader. State: Replica: 9260a93630614775965df8b88ab92d7f, State: Running, Role: LEADER
I20260810 19:37:14.206992 20723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260a93630614775965df8b88ab92d7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45439 } }
I20260810 19:37:14.210681 20725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9260a93630614775965df8b88ab92d7f. Latest consensus state: current_term: 1 leader_uuid: "9260a93630614775965df8b88ab92d7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260a93630614775965df8b88ab92d7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45439 } } }
I20260810 19:37:14.210759 20724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9260a93630614775965df8b88ab92d7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9260a93630614775965df8b88ab92d7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45439 } } }
I20260810 19:37:14.211145 20724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:14.211145 20725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:14.215036 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:14.197745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35259 (local address 127.0.0.1:45439)
0810 19:37:14.197961 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:14.197967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:14.198504 (+ 537us) server_negotiation.cc:407] Connection header received
0810 19:37:14.198615 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:14.198620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:14.198715 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:14.198863 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:14.200815 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:14.201706 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:14.203036 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:14.203500 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:14.203637 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:14.206214 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:14.206220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:14.206223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:14.206255 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:14.210040 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:14.211213 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:14.211218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:14.211221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:14.211385 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:14.212005 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:14.212009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:14.212012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:14.212433 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:14.214008 (+ 1575us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:14.214233 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 19:37:14.214514 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 19:37:14.219507 20731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:14.224567 20731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:14.233366 20731 catalog_manager.cc:1347] Generated new cluster ID: a03dd30bf7fc4fdeb3ff230fd4dd8e2a
I20260810 19:37:14.233515 20731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:14.244029 20731 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:14.245704 20731 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:14.260864 20731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9260a93630614775965df8b88ab92d7f: Generated new TSK 0
I20260810 19:37:14.262020 20731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:14.268947 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20640
I20260810 19:37:14.270404 20707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:14.363611 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20640
W20260810 19:37:14.384737 20625 connection.cc:535] client connection to 127.0.0.1:45439 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45439): Connection reset by peer (error 104)
2026-08-10T19:37:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2187 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:14 dist-test-slave-tm1g krb5kdc[20746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:14 dist-test-slave-tm1g krb5kdc[20746](info): set up 2 sockets
Aug 10 19:37:14 dist-test-slave-tm1g krb5kdc[20746](info): commencing operation
krb5kdc: starting...
W20260810 19:37:15.551347 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:15 dist-test-slave-tm1g krb5kdc[20746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:15.877861 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:16.225875 20760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:16.226248 20760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:16.226454 20760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:16.233863 20760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:16.234040 20760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:16.234151 20760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:16.234262 20760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:16.234359 20760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:16.244567 20760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:16.246013 20760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:16.247452 20760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:16.259773 20766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:16.259754 20767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:16.260059 20769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:16.260514 20760 server_base.cc:1034] running on GCE node
I20260810 19:37:16.261323 20760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:16.262861 20760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:16.264115 20760 hybrid_clock.cc:648] HybridClock initialized: now 1786390636264071 us; error 45 us; skew 500 ppm
I20260810 19:37:16.264679 20760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:16 dist-test-slave-tm1g krb5kdc[20746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390636, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:16.268226 20760 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:16.269872 20760 webserver.cc:460] Webserver started at http://127.0.0.1:33009/ using document root <none> and password file <none>
I20260810 19:37:16.270668 20760 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:16.270804 20760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:16.271142 20760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:16.274241 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd"
format_stamp: "Formatted at 2026-08-10 19:37:16 on dist-test-slave-tm1g"
server_key: "447fe6cebac49327741400e16c26b8e5"
server_key_iv: "a3149cef17a53116c6e52e9ca7625be4"
server_key_version: "encryptionkey@0"
I20260810 19:37:16.275098 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd"
format_stamp: "Formatted at 2026-08-10 19:37:16 on dist-test-slave-tm1g"
server_key: "447fe6cebac49327741400e16c26b8e5"
server_key_iv: "a3149cef17a53116c6e52e9ca7625be4"
server_key_version: "encryptionkey@0"
I20260810 19:37:16.280498 20760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:37:16.284046 20776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:16.285447 20760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 19:37:16.285633 20760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd"
format_stamp: "Formatted at 2026-08-10 19:37:16 on dist-test-slave-tm1g"
server_key: "447fe6cebac49327741400e16c26b8e5"
server_key_iv: "a3149cef17a53116c6e52e9ca7625be4"
server_key_version: "encryptionkey@0"
I20260810 19:37:16.285806 20760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:16.309769 20760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:16.314791 20760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:16.315104 20760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:16.334715 20760 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33751
I20260810 19:37:16.334758 20832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33751 every 8 connection(s)
I20260810 19:37:16.336186 20760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:16.340160 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:16.341120 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20760
I20260810 19:37:16.341328 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:16.341919 30630 external_mini_cluster.cc:1428] Setting key 6e55cce490eeb90d5e3e2acb460c92cf
I20260810 19:37:16.354888 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd: Bootstrap starting.
I20260810 19:37:16.356294 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:16.343922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54149 (local address 127.0.0.1:33751)
0810 19:37:16.344750 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:16.344765 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:16.344811 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:37:16.346777 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:16.346805 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:16.347237 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:16.347580 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:16.348232 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:16.349529 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:16.351547 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:16.352312 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:16.352448 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:16.353813 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:16.353820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:16.353836 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:16.354550 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:16.354635 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:16.354661 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:16.354794 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:16.355162 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 19:37:16.355306 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":378,"threads_started":1}
I20260810 19:37:16.358664 20833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:16.360041 20833 log.cc:824] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:16.360563 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:16.363125 20833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd: No bootstrap required, opened a new log
I20260810 19:37:16.372452 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:16.365392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44987 (local address 127.0.0.1:33751)
0810 19:37:16.365601 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:16.365607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:16.366098 (+ 491us) server_negotiation.cc:407] Connection header received
0810 19:37:16.366242 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:16.366247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:16.366341 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:16.366469 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:16.366926 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:16.368086 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:16.369480 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:16.370182 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:16.370330 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:16.371653 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:16.371660 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:16.371666 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:16.371689 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:16.371743 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:16.371770 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:16.371906 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:16.372048 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 19:37:16.372109 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:37:16.373351 20833 raft_consensus.cc:348] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33751 } }
I20260810 19:37:16.373917 20833 raft_consensus.cc:374] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:16.374039 20833 raft_consensus.cc:729] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26bfb7820cf49c1ab9a8f6cf89d4bcd, State: Initialized, Role: FOLLOWER
I20260810 19:37:16.374703 20833 consensus_queue.cc:260] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33751 } }
I20260810 19:37:16.374955 20833 raft_consensus.cc:388] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:16.375108 20833 raft_consensus.cc:482] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:16.375295 20833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:16.377537 20833 raft_consensus.cc:504] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33751 } }
I20260810 19:37:16.378048 20833 leader_election.cc:302] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e26bfb7820cf49c1ab9a8f6cf89d4bcd; no voters:
I20260810 19:37:16.378691 20833 leader_election.cc:288] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:16.378827 20843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:16.380450 20843 raft_consensus.cc:686] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [term 1 LEADER]: Becoming Leader. State: Replica: e26bfb7820cf49c1ab9a8f6cf89d4bcd, State: Running, Role: LEADER
I20260810 19:37:16.380965 20833 sys_catalog.cc:564] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:16.381192 20843 consensus_queue.cc:237] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33751 } }
I20260810 19:37:16.384297 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e26bfb7820cf49c1ab9a8f6cf89d4bcd. Latest consensus state: current_term: 1 leader_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33751 } } }
I20260810 19:37:16.384744 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:16.384402 20844 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26bfb7820cf49c1ab9a8f6cf89d4bcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33751 } } }
I20260810 19:37:16.384902 20844 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:16.387247 20854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:16.393180 20854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:16.401528 20854 catalog_manager.cc:1347] Generated new cluster ID: 1d17f2d51b514f64bf9b67e979a58ef6
I20260810 19:37:16.401651 20854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:16.427348 20854 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:16.429469 20854 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:16.439091 20854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e26bfb7820cf49c1ab9a8f6cf89d4bcd: Generated new TSK 0
I20260810 19:37:16.440198 20854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:16.459007 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20760
W20260810 19:37:16.460747 20827 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54149
W20260810 19:37:16.461556 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20760: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:16.461751 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20760
W20260810 19:37:16.483431 20744 connection.cc:535] client connection to 127.0.0.1:33751 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33751): Connection reset by peer (error 104)
2026-08-10T19:37:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2097 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:16 dist-test-slave-tm1g krb5kdc[20868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:16 dist-test-slave-tm1g krb5kdc[20868](info): set up 2 sockets
Aug 10 19:37:16 dist-test-slave-tm1g krb5kdc[20868](info): commencing operation
krb5kdc: starting...
W20260810 19:37:17.654711 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:17 dist-test-slave-tm1g krb5kdc[20868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:18.006775 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:18.359139 20881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:18.359508 20881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:18.359678 20881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:18.367566 20881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:18.367722 20881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:18.367825 20881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:18.367933 20881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:18.368010 20881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:18.378082 20881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38273
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:18.379484 20881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:18.380820 20881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:18.392877 20890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:18.392963 20887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:18.392957 20888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:18.393724 20881 server_base.cc:1034] running on GCE node
I20260810 19:37:18.394613 20881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:18.396111 20881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:18.397353 20881 hybrid_clock.cc:648] HybridClock initialized: now 1786390638397314 us; error 54 us; skew 500 ppm
I20260810 19:37:18.397724 20881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:18 dist-test-slave-tm1g krb5kdc[20868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:18.400942 20881 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:18.402570 20881 webserver.cc:460] Webserver started at http://127.0.0.1:44071/ using document root <none> and password file <none>
I20260810 19:37:18.403275 20881 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:18.403430 20881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:18.403759 20881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:18.407020 20881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "e485c2f60e7047549fed8bbfc1bbb116"
format_stamp: "Formatted at 2026-08-10 19:37:18 on dist-test-slave-tm1g"
server_key: "14c937e79ec7fefc2a6130da6888a713"
server_key_iv: "9a79faf7752f9fc914fc8a24d35fa5a4"
server_key_version: "encryptionkey@0"
I20260810 19:37:18.407828 20881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "e485c2f60e7047549fed8bbfc1bbb116"
format_stamp: "Formatted at 2026-08-10 19:37:18 on dist-test-slave-tm1g"
server_key: "14c937e79ec7fefc2a6130da6888a713"
server_key_iv: "9a79faf7752f9fc914fc8a24d35fa5a4"
server_key_version: "encryptionkey@0"
I20260810 19:37:18.413466 20881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:37:18.417764 20897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:18.419127 20881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:37:18.419320 20881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "e485c2f60e7047549fed8bbfc1bbb116"
format_stamp: "Formatted at 2026-08-10 19:37:18 on dist-test-slave-tm1g"
server_key: "14c937e79ec7fefc2a6130da6888a713"
server_key_iv: "9a79faf7752f9fc914fc8a24d35fa5a4"
server_key_version: "encryptionkey@0"
I20260810 19:37:18.419548 20881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:18.425669 20881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:18.429878 20881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:18.430192 20881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:18.450189 20881 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38273
I20260810 19:37:18.450235 20951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38273 every 8 connection(s)
I20260810 19:37:18.451695 20881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:18.453941 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20881
I20260810 19:37:18.454164 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:18.454675 30630 external_mini_cluster.cc:1428] Setting key 3ee31dcdb4edd4d6004b1af042a28d39
I20260810 19:37:18.455969 20952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:18.463379 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:18.456574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36867 (local address 127.0.0.1:38273)
0810 19:37:18.457440 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:18.457457 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:18.457509 (+ 52us) server_negotiation.cc:407] Connection header received
0810 19:37:18.459436 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:18.459462 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:18.459917 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:18.460354 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:18.461347 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:18.461355 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:18.461371 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:18.461389 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:18.461488 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:18.461526 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:18.461754 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:18.461904 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 19:37:18.462110 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":376,"threads_started":1}
I20260810 19:37:18.467103 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:18.470922 20952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116: Bootstrap starting.
I20260810 19:37:18.475261 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:18.472503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45489 (local address 127.0.0.1:38273)
0810 19:37:18.472740 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:18.472749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:18.473198 (+ 449us) server_negotiation.cc:407] Connection header received
0810 19:37:18.473359 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:18.473362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:18.473412 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:18.473491 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:18.474581 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:18.474585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:18.474588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:18.474602 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:18.474641 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:18.474656 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:18.474741 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:18.474890 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 19:37:18.474938 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 19:37:18.475718 20952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:18.476898 20952 log.cc:824] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:18.479652 20952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116: No bootstrap required, opened a new log
I20260810 19:37:18.487008 20952 raft_consensus.cc:348] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e485c2f60e7047549fed8bbfc1bbb116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38273 } }
I20260810 19:37:18.487460 20952 raft_consensus.cc:374] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:18.487617 20952 raft_consensus.cc:729] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e485c2f60e7047549fed8bbfc1bbb116, State: Initialized, Role: FOLLOWER
I20260810 19:37:18.488166 20952 consensus_queue.cc:260] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e485c2f60e7047549fed8bbfc1bbb116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38273 } }
I20260810 19:37:18.488423 20952 raft_consensus.cc:388] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:18.488579 20952 raft_consensus.cc:482] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:18.488749 20952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:18.490988 20952 raft_consensus.cc:504] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e485c2f60e7047549fed8bbfc1bbb116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38273 } }
I20260810 19:37:18.491499 20952 leader_election.cc:302] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e485c2f60e7047549fed8bbfc1bbb116; no voters:
I20260810 19:37:18.492106 20952 leader_election.cc:288] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:18.492261 20962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:18.493028 20962 raft_consensus.cc:686] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [term 1 LEADER]: Becoming Leader. State: Replica: e485c2f60e7047549fed8bbfc1bbb116, State: Running, Role: LEADER
I20260810 19:37:18.493607 20962 consensus_queue.cc:237] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e485c2f60e7047549fed8bbfc1bbb116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38273 } }
I20260810 19:37:18.494371 20952 sys_catalog.cc:564] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:18.497010 20963 sys_catalog.cc:455] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e485c2f60e7047549fed8bbfc1bbb116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e485c2f60e7047549fed8bbfc1bbb116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38273 } } }
I20260810 19:37:18.497507 20963 sys_catalog.cc:458] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:18.497040 20964 sys_catalog.cc:455] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e485c2f60e7047549fed8bbfc1bbb116. Latest consensus state: current_term: 1 leader_uuid: "e485c2f60e7047549fed8bbfc1bbb116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e485c2f60e7047549fed8bbfc1bbb116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38273 } } }
I20260810 19:37:18.501535 20964 sys_catalog.cc:458] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:18.502575 20972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:18.507200 20972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:18.515726 20972 catalog_manager.cc:1347] Generated new cluster ID: 6f456a1329a54fa0aa80862e0979e5a9
I20260810 19:37:18.515866 20972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:18.531670 20972 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:18.532167 20972 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:18.542550 20972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e485c2f60e7047549fed8bbfc1bbb116: Generated new TSK 0
I20260810 19:37:18.543372 20972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:18.548405 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20881
W20260810 19:37:18.549721 20946 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36867
W20260810 19:37:18.550572 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20881: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:18.550735 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20881
2026-08-10T19:37:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2089 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:18 dist-test-slave-tm1g krb5kdc[20985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:18 dist-test-slave-tm1g krb5kdc[20985](info): set up 2 sockets
Aug 10 19:37:18 dist-test-slave-tm1g krb5kdc[20985](info): commencing operation
krb5kdc: starting...
W20260810 19:37:19.749197 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:19 dist-test-slave-tm1g krb5kdc[20985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:20.085211 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33375
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:20.438007 20999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:20.438304 20999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:20.438484 20999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:20.445739 20999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:20.445845 20999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:20.445899 20999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:20.445962 20999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:20.446009 20999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:20.455945 20999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:20.457230 20999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:20.458636 20999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:20.470662 21005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:20.470758 21004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:20.470733 21007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:20.471905 20999 server_base.cc:1034] running on GCE node
I20260810 19:37:20.472711 20999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:20.474123 20999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:20.475363 20999 hybrid_clock.cc:648] HybridClock initialized: now 1786390640475325 us; error 45 us; skew 500 ppm
I20260810 19:37:20.475728 20999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:20 dist-test-slave-tm1g krb5kdc[20985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:20.484457 20999 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:20.486055 20999 webserver.cc:460] Webserver started at http://127.0.0.1:37003/ using document root <none> and password file <none>
I20260810 19:37:20.486815 20999 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:20.486931 20999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:20.487206 20999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:20.490286 20999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "dfd2d691c0d84b01abbeeae99f73d6aa"
format_stamp: "Formatted at 2026-08-10 19:37:20 on dist-test-slave-tm1g"
server_key: "e6a285ab42a0e6a913820860a220ab1e"
server_key_iv: "6a176bd3f5ff3b50ced6e18ab1e9507f"
server_key_version: "encryptionkey@0"
I20260810 19:37:20.491117 20999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "dfd2d691c0d84b01abbeeae99f73d6aa"
format_stamp: "Formatted at 2026-08-10 19:37:20 on dist-test-slave-tm1g"
server_key: "e6a285ab42a0e6a913820860a220ab1e"
server_key_iv: "6a176bd3f5ff3b50ced6e18ab1e9507f"
server_key_version: "encryptionkey@0"
I20260810 19:37:20.496276 20999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:37:20.499698 21015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:20.501128 20999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:20.501308 20999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "dfd2d691c0d84b01abbeeae99f73d6aa"
format_stamp: "Formatted at 2026-08-10 19:37:20 on dist-test-slave-tm1g"
server_key: "e6a285ab42a0e6a913820860a220ab1e"
server_key_iv: "6a176bd3f5ff3b50ced6e18ab1e9507f"
server_key_version: "encryptionkey@0"
I20260810 19:37:20.501511 20999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:20.529795 20999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:20.534502 20999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:20.534818 20999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:20.554098 20999 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33375
I20260810 19:37:20.554142 21070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33375 every 8 connection(s)
I20260810 19:37:20.555572 20999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:20.559386 21071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:20.561300 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 20999
I20260810 19:37:20.561519 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:20.562106 30630 external_mini_cluster.cc:1428] Setting key cc88af81688acc8339a8224a880a8134
I20260810 19:37:20.573859 21071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa: Bootstrap starting.
Aug 10 19:37:20 dist-test-slave-tm1g krb5kdc[20985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 19:37:20.577881 21071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:20.579031 21071 log.cc:824] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:20.582152 21071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa: No bootstrap required, opened a new log
I20260810 19:37:20.586103 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:20.563997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53871 (local address 127.0.0.1:33375)
0810 19:37:20.564860 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:20.564871 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:20.564911 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:37:20.566630 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:20.566656 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:20.567061 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:20.567386 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:20.569244 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:20.570650 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:20.572536 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:20.573200 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:20.573326 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:20.578505 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:20.578513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:20.578528 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:20.578568 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:20.581758 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:20.582649 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:20.582656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:20.582662 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:20.582836 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:20.583252 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:20.583257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:20.583262 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:20.583815 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:20.584156 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:20.584620 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 19:37:20.585061 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":410,"threads_started":1}
I20260810 19:37:20.589888 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:20.590433 21071 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260810 19:37:20.590802 21071 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:20.590920 21071 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfd2d691c0d84b01abbeeae99f73d6aa, State: Initialized, Role: FOLLOWER
I20260810 19:37:20.591461 21071 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260810 19:37:20.591711 21071 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:20.591859 21071 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:20.592046 21071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:20.594442 21071 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260810 19:37:20.595053 21071 leader_election.cc:302] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfd2d691c0d84b01abbeeae99f73d6aa; no voters:
I20260810 19:37:20.595746 21071 leader_election.cc:288] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:20.595872 21082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:20.597834 21071 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:20.598438 21082 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [term 1 LEADER]: Becoming Leader. State: Replica: dfd2d691c0d84b01abbeeae99f73d6aa, State: Running, Role: LEADER
I20260810 19:37:20.599128 21082 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } }
I20260810 19:37:20.602473 21083 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } } }
I20260810 19:37:20.602979 21083 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:20.604822 21084 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfd2d691c0d84b01abbeeae99f73d6aa. Latest consensus state: current_term: 1 leader_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd2d691c0d84b01abbeeae99f73d6aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33375 } } }
I20260810 19:37:20.606680 21084 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:20.614914 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:20.594784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33745 (local address 127.0.0.1:33375)
0810 19:37:20.594975 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:20.594981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:20.595401 (+ 420us) server_negotiation.cc:407] Connection header received
0810 19:37:20.595586 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:20.595594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:20.595690 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:20.595813 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:20.598140 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:20.599423 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:20.601267 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:20.602161 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:20.602318 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:20.606606 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:20.606612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:20.606617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:20.606660 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:20.611157 (+ 4497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:20.611938 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:20.611946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:20.611953 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:20.612145 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:20.612715 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:20.612722 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:20.612728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:20.613287 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:20.613615 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:20.614089 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 19:37:20.614417 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 19:37:20.617172 21090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:20.622520 21090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:20.631284 21090 catalog_manager.cc:1347] Generated new cluster ID: 5211c06edd734680a9eb836b38922f5c
I20260810 19:37:20.631402 21090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:20.654032 21090 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:20.655500 21090 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:20.665273 21090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfd2d691c0d84b01abbeeae99f73d6aa: Generated new TSK 0
I20260810 19:37:20.666296 21090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:20.677210 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 20999
I20260810 19:37:20.678445 21065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:20.771606 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 20999
2026-08-10T19:37:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2219 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:20 dist-test-slave-tm1g krb5kdc[21105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:20 dist-test-slave-tm1g krb5kdc[21105](info): set up 2 sockets
Aug 10 19:37:20 dist-test-slave-tm1g krb5kdc[21105](info): commencing operation
krb5kdc: starting...
W20260810 19:37:21.962749 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:22 dist-test-slave-tm1g krb5kdc[21105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:22.297591 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:22.650581 21120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:22.650861 21120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:22.651064 21120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:22.658149 21120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:22.658298 21120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:22.658424 21120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:22.658535 21120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:22.658604 21120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:22.668653 21120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:22.670127 21120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:22.671572 21120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:22.684016 21126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:22.683969 21128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:22.683962 21125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:22.685266 21120 server_base.cc:1034] running on GCE node
I20260810 19:37:22.686169 21120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:22.687637 21120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:22.688840 21120 hybrid_clock.cc:648] HybridClock initialized: now 1786390642688808 us; error 30 us; skew 500 ppm
I20260810 19:37:22.689201 21120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:22 dist-test-slave-tm1g krb5kdc[21105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390642, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:22.692408 21120 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:22.693935 21120 webserver.cc:460] Webserver started at http://127.0.0.1:45601/ using document root <none> and password file <none>
I20260810 19:37:22.694712 21120 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:22.694866 21120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:22.695189 21120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:22.698172 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "57caf2649de145ff9885eec5ab10153d"
format_stamp: "Formatted at 2026-08-10 19:37:22 on dist-test-slave-tm1g"
server_key: "ad29d33f5784e5f2a0b2fd19e72a2c68"
server_key_iv: "4d620b0a728f49f2f4559fcc70362f51"
server_key_version: "encryptionkey@0"
I20260810 19:37:22.699030 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "57caf2649de145ff9885eec5ab10153d"
format_stamp: "Formatted at 2026-08-10 19:37:22 on dist-test-slave-tm1g"
server_key: "ad29d33f5784e5f2a0b2fd19e72a2c68"
server_key_iv: "4d620b0a728f49f2f4559fcc70362f51"
server_key_version: "encryptionkey@0"
I20260810 19:37:22.704156 21120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 19:37:22.707671 21136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:22.708895 21120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:22.709088 21120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "57caf2649de145ff9885eec5ab10153d"
format_stamp: "Formatted at 2026-08-10 19:37:22 on dist-test-slave-tm1g"
server_key: "ad29d33f5784e5f2a0b2fd19e72a2c68"
server_key_iv: "4d620b0a728f49f2f4559fcc70362f51"
server_key_version: "encryptionkey@0"
I20260810 19:37:22.709288 21120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:22.739531 21120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:22.744151 21120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:22.744398 21120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:22.763824 21120 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44565
I20260810 19:37:22.763856 21191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44565 every 8 connection(s)
I20260810 19:37:22.765312 21120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:22.769178 21192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:22.771641 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21120
I20260810 19:37:22.771832 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:22.772362 30630 external_mini_cluster.cc:1428] Setting key 8703f9157daecfd88a98d733cd000642
I20260810 19:37:22.781804 21192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d: Bootstrap starting.
I20260810 19:37:22.785549 21192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:22.785957 21195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:22.774239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35413 (local address 127.0.0.1:44565)
0810 19:37:22.775101 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:22.775112 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:22.775159 (+ 47us) server_negotiation.cc:407] Connection header received
0810 19:37:22.776840 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:22.776862 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:22.777244 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:22.777564 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:22.778198 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:22.779635 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:22.781394 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:22.782119 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:22.782248 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:22.783568 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:22.783577 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:22.783593 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:22.784119 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:22.784194 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:22.784218 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:22.784312 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:22.784546 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 19:37:22.784844 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":385,"threads_started":1}
I20260810 19:37:22.786950 21192 log.cc:824] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:22.790125 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:22.790361 21192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d: No bootstrap required, opened a new log
I20260810 19:37:22.798249 21192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57caf2649de145ff9885eec5ab10153d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44565 } }
I20260810 19:37:22.798739 21192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:22.798869 21192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57caf2649de145ff9885eec5ab10153d, State: Initialized, Role: FOLLOWER
I20260810 19:37:22.799513 21192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57caf2649de145ff9885eec5ab10153d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44565 } }
I20260810 19:37:22.799818 21192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:22.799973 21192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:22.800186 21192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:22.801090 21195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:22.794692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43969 (local address 127.0.0.1:44565)
0810 19:37:22.794885 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:22.794892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:22.795250 (+ 358us) server_negotiation.cc:407] Connection header received
0810 19:37:22.795474 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:22.795479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:22.795552 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:22.795668 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:22.796201 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:22.796979 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:22.798334 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:22.798924 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:22.799067 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:22.800290 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:22.800296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:22.800299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:22.800337 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:22.800385 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:22.800405 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:22.800513 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:22.800673 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 19:37:22.800773 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 19:37:22.803360 21192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57caf2649de145ff9885eec5ab10153d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44565 } }
I20260810 19:37:22.803877 21192 leader_election.cc:302] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57caf2649de145ff9885eec5ab10153d; no voters:
I20260810 19:37:22.804454 21192 leader_election.cc:288] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:22.804618 21202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:22.805346 21202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [term 1 LEADER]: Becoming Leader. State: Replica: 57caf2649de145ff9885eec5ab10153d, State: Running, Role: LEADER
I20260810 19:37:22.805930 21202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57caf2649de145ff9885eec5ab10153d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44565 } }
I20260810 19:37:22.807109 21192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:22.809064 21203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57caf2649de145ff9885eec5ab10153d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57caf2649de145ff9885eec5ab10153d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44565 } } }
I20260810 19:37:22.809109 21204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57caf2649de145ff9885eec5ab10153d. Latest consensus state: current_term: 1 leader_uuid: "57caf2649de145ff9885eec5ab10153d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57caf2649de145ff9885eec5ab10153d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44565 } } }
I20260810 19:37:22.809517 21204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:22.809517 21203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:22.818981 21209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:22.823453 21209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:22.832612 21209 catalog_manager.cc:1347] Generated new cluster ID: ea921f1f302c447ead82fd706c83001b
I20260810 19:37:22.832751 21209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:22.850225 21209 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:22.851696 21209 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:22.858587 21209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57caf2649de145ff9885eec5ab10153d: Generated new TSK 0
I20260810 19:37:22.859460 21209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:22.870180 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21120
W20260810 19:37:22.871417 21187 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35413
W20260810 19:37:22.872200 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21120: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:22.872370 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21120
2026-08-10T19:37:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2101 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:22 dist-test-slave-tm1g krb5kdc[21225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:22 dist-test-slave-tm1g krb5kdc[21225](info): set up 2 sockets
Aug 10 19:37:22 dist-test-slave-tm1g krb5kdc[21225](info): commencing operation
krb5kdc: starting...
W20260810 19:37:24.058215 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:24 dist-test-slave-tm1g krb5kdc[21225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:24.395272 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:24.739302 21238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:24.739640 21238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:24.739861 21238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:24.747115 21238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:24.747263 21238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:24.747359 21238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:24.747471 21238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:24.747546 21238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:24.757370 21238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:24.758746 21238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:24.760097 21238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:24.771979 21244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:24.772001 21247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:24.772292 21243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:24.772980 21238 server_base.cc:1034] running on GCE node
I20260810 19:37:24.773715 21238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:24.775111 21238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:24.776356 21238 hybrid_clock.cc:648] HybridClock initialized: now 1786390644776303 us; error 47 us; skew 500 ppm
I20260810 19:37:24.776706 21238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:24 dist-test-slave-tm1g krb5kdc[21225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:24.779896 21238 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:24.781332 21238 webserver.cc:460] Webserver started at http://127.0.0.1:40585/ using document root <none> and password file <none>
I20260810 19:37:24.781991 21238 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:24.782138 21238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:24.782521 21238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:24.785485 21238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "d1bf857a5a5849e1ba3a2a9d8323d549"
format_stamp: "Formatted at 2026-08-10 19:37:24 on dist-test-slave-tm1g"
server_key: "25d8d430777c465991bc45fb86818bd5"
server_key_iv: "a047ca539c178c3cf1bb8b1ea4f59c93"
server_key_version: "encryptionkey@0"
I20260810 19:37:24.786293 21238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "d1bf857a5a5849e1ba3a2a9d8323d549"
format_stamp: "Formatted at 2026-08-10 19:37:24 on dist-test-slave-tm1g"
server_key: "25d8d430777c465991bc45fb86818bd5"
server_key_iv: "a047ca539c178c3cf1bb8b1ea4f59c93"
server_key_version: "encryptionkey@0"
I20260810 19:37:24.791419 21238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:37:24.794795 21255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:24.796093 21238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:37:24.796268 21238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "d1bf857a5a5849e1ba3a2a9d8323d549"
format_stamp: "Formatted at 2026-08-10 19:37:24 on dist-test-slave-tm1g"
server_key: "25d8d430777c465991bc45fb86818bd5"
server_key_iv: "a047ca539c178c3cf1bb8b1ea4f59c93"
server_key_version: "encryptionkey@0"
I20260810 19:37:24.796475 21238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:24.802091 21238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:24.806376 21238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:24.806689 21238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:24.825747 21238 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46227
I20260810 19:37:24.825794 21309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46227 every 8 connection(s)
I20260810 19:37:24.827148 21238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:24.830965 21310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:24.836792 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21238
I20260810 19:37:24.837018 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:24.837479 30630 external_mini_cluster.cc:1428] Setting key 0ff2fe1a5d566c73bb966fd1acaba1ff
I20260810 19:37:24.842162 21310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549: Bootstrap starting.
I20260810 19:37:24.845710 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:24.839669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39791 (local address 127.0.0.1:46227)
0810 19:37:24.840547 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:24.840559 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:24.840600 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:37:24.842257 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:24.842278 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:24.842686 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:24.843010 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:24.844178 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:24.844184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:24.844199 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:24.844213 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:24.844278 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:24.844299 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:24.844466 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:24.844608 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 19:37:24.844803 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":419,"threads_started":1}
I20260810 19:37:24.846051 21310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:24.847427 21310 log.cc:824] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:24.849968 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:24.850674 21310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549: No bootstrap required, opened a new log
I20260810 19:37:24.857465 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:24.854553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54457 (local address 127.0.0.1:46227)
0810 19:37:24.854822 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:24.854835 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:24.855386 (+ 551us) server_negotiation.cc:407] Connection header received
0810 19:37:24.855531 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:24.855536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:24.855594 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:24.855711 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:24.856699 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:24.856705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:24.856710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:24.856728 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:24.856784 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:24.856804 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:24.856926 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:24.857097 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 19:37:24.857159 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:37:24.859453 21310 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260810 19:37:24.859782 21310 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:24.859865 21310 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bf857a5a5849e1ba3a2a9d8323d549, State: Initialized, Role: FOLLOWER
I20260810 19:37:24.860348 21310 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260810 19:37:24.860569 21310 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:24.860666 21310 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:24.860790 21310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:24.863071 21310 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260810 19:37:24.863535 21310 leader_election.cc:302] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1bf857a5a5849e1ba3a2a9d8323d549; no voters:
I20260810 19:37:24.864151 21310 leader_election.cc:288] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:24.864255 21321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:24.865729 21321 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [term 1 LEADER]: Becoming Leader. State: Replica: d1bf857a5a5849e1ba3a2a9d8323d549, State: Running, Role: LEADER
I20260810 19:37:24.866237 21321 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260810 19:37:24.866456 21310 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:24.870463 21322 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } } }
I20260810 19:37:24.870430 21323 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1bf857a5a5849e1ba3a2a9d8323d549. Latest consensus state: current_term: 1 leader_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bf857a5a5849e1ba3a2a9d8323d549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } } }
I20260810 19:37:24.871112 21322 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:24.871126 21323 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:24.875029 21329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:24.882061 21329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:24.894631 21329 catalog_manager.cc:1347] Generated new cluster ID: 59307969dad34af8adcb7e2b7a98adb0
I20260810 19:37:24.894737 21329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:24.935055 21329 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:24.935528 21329 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:24.955629 21329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1bf857a5a5849e1ba3a2a9d8323d549: Generated new TSK 0
I20260810 19:37:24.956519 21329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:24.966895 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21238
W20260810 19:37:24.968024 21305 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39791
W20260810 19:37:24.968713 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21238: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:24.968858 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21238
2026-08-10T19:37:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2095 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:25 dist-test-slave-tm1g krb5kdc[21342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:25 dist-test-slave-tm1g krb5kdc[21342](info): set up 2 sockets
Aug 10 19:37:25 dist-test-slave-tm1g krb5kdc[21342](info): commencing operation
krb5kdc: starting...
W20260810 19:37:26.154973 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:26 dist-test-slave-tm1g krb5kdc[21342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:26.477329 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:26.816994 21355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:26.817322 21355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:26.817414 21355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:26.824724 21355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:26.824847 21355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:26.824894 21355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:26.824963 21355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:26.825008 21355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:26.835093 21355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:26.836587 21355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:26.838238 21355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:26.850672 21360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:26.850785 21361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:26.850704 21363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:26.855777 21355 server_base.cc:1034] running on GCE node
I20260810 19:37:26.856668 21355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:26.858122 21355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:26.859366 21355 hybrid_clock.cc:648] HybridClock initialized: now 1786390646859325 us; error 56 us; skew 500 ppm
I20260810 19:37:26.859758 21355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:26 dist-test-slave-tm1g krb5kdc[21342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:26.870572 21355 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:26.872205 21355 webserver.cc:460] Webserver started at http://127.0.0.1:43739/ using document root <none> and password file <none>
I20260810 19:37:26.872949 21355 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:26.873102 21355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:26.873414 21355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:26.876605 21355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "2a81f20dc9ea47769b1a005c2c00d668"
format_stamp: "Formatted at 2026-08-10 19:37:26 on dist-test-slave-tm1g"
server_key: "8c78c3c558b18d6c9e1291e3466edc08"
server_key_iv: "40d2cf665e67c9b12ba8625a1c5fa7c9"
server_key_version: "encryptionkey@0"
I20260810 19:37:26.877398 21355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "2a81f20dc9ea47769b1a005c2c00d668"
format_stamp: "Formatted at 2026-08-10 19:37:26 on dist-test-slave-tm1g"
server_key: "8c78c3c558b18d6c9e1291e3466edc08"
server_key_iv: "40d2cf665e67c9b12ba8625a1c5fa7c9"
server_key_version: "encryptionkey@0"
I20260810 19:37:26.882697 21355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 19:37:26.886052 21370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:26.887369 21355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:37:26.887568 21355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "2a81f20dc9ea47769b1a005c2c00d668"
format_stamp: "Formatted at 2026-08-10 19:37:26 on dist-test-slave-tm1g"
server_key: "8c78c3c558b18d6c9e1291e3466edc08"
server_key_iv: "40d2cf665e67c9b12ba8625a1c5fa7c9"
server_key_version: "encryptionkey@0"
I20260810 19:37:26.887769 21355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:26.907793 21355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:26.912473 21355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:26.912770 21355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:26.932183 21355 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43729
I20260810 19:37:26.932217 21425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43729 every 8 connection(s)
I20260810 19:37:26.933559 21355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:26.937225 21426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:26.937487 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21355
I20260810 19:37:26.937667 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:26.938150 30630 external_mini_cluster.cc:1428] Setting key a652e9ef729ba746b438bbc96c44f622
I20260810 19:37:26.951105 21426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668: Bootstrap starting.
Aug 10 19:37:26 dist-test-slave-tm1g krb5kdc[21342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 19:37:26.954823 21426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:26.955938 21426 log.cc:824] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:26.958837 21426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668: No bootstrap required, opened a new log
I20260810 19:37:26.962178 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:26.939924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:45891 (local address 127.0.0.1:43729)
0810 19:37:26.940840 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:26.940851 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:26.940892 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:37:26.943196 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:26.943217 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:26.943616 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:26.943988 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:26.945772 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:26.947142 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:26.949030 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:26.949690 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:26.949830 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:26.954740 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:26.954750 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:26.954763 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:26.954801 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:26.957965 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:26.958834 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:26.958840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:26.958846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:26.959047 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:26.959520 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:26.959525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:26.959530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:26.960027 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:26.960384 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:26.960892 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 19:37:26.961211 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":458,"threads_started":1}
I20260810 19:37:26.965945 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:26.967020 21426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a81f20dc9ea47769b1a005c2c00d668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43729 } }
I20260810 19:37:26.967388 21426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:26.967501 21426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a81f20dc9ea47769b1a005c2c00d668, State: Initialized, Role: FOLLOWER
I20260810 19:37:26.968022 21426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a81f20dc9ea47769b1a005c2c00d668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43729 } }
I20260810 19:37:26.968266 21426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:26.968410 21426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:26.968596 21426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:26.971450 21426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a81f20dc9ea47769b1a005c2c00d668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43729 } }
I20260810 19:37:26.971956 21426 leader_election.cc:302] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a81f20dc9ea47769b1a005c2c00d668; no voters:
I20260810 19:37:26.972627 21426 leader_election.cc:288] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:26.972728 21439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:26.975255 21426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:26.979480 21439 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [term 1 LEADER]: Becoming Leader. State: Replica: 2a81f20dc9ea47769b1a005c2c00d668, State: Running, Role: LEADER
I20260810 19:37:26.980222 21439 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a81f20dc9ea47769b1a005c2c00d668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43729 } }
I20260810 19:37:26.983845 21440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a81f20dc9ea47769b1a005c2c00d668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a81f20dc9ea47769b1a005c2c00d668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43729 } } }
I20260810 19:37:26.983866 21441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a81f20dc9ea47769b1a005c2c00d668. Latest consensus state: current_term: 1 leader_uuid: "2a81f20dc9ea47769b1a005c2c00d668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a81f20dc9ea47769b1a005c2c00d668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43729 } } }
I20260810 19:37:26.984387 21440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:26.984386 21441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:26.987519 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:26.970739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:52397 (local address 127.0.0.1:43729)
0810 19:37:26.970908 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:26.970915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:26.971320 (+ 405us) server_negotiation.cc:407] Connection header received
0810 19:37:26.971524 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:26.971529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:26.971625 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:26.971717 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:26.973369 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:26.974354 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:26.975716 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:26.976152 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:26.976303 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:26.979271 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:26.979276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:26.979279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:26.979307 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:26.982678 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:26.983791 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:26.983797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:26.983804 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:26.984028 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:26.984701 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:26.984708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:26.984718 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:26.985181 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:26.986541 (+ 1360us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:26.986770 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 19:37:26.987021 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 19:37:26.996289 21449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:27.000763 21449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:27.008980 21449 catalog_manager.cc:1347] Generated new cluster ID: 95c03efe2a034064ae8c21af42ec31d4
I20260810 19:37:27.009186 21449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:27.024896 21449 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:27.026226 21449 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:27.032668 21449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a81f20dc9ea47769b1a005c2c00d668: Generated new TSK 0
I20260810 19:37:27.033480 21449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:27.043098 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21355
I20260810 19:37:27.045452 21421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:27.136447 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21355
W20260810 19:37:27.157357 21340 connection.cc:535] client connection to 127.0.0.1:43729 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43729): Connection reset by peer (error 104)
2026-08-10T19:37:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2167 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:27 dist-test-slave-tm1g krb5kdc[21463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:27 dist-test-slave-tm1g krb5kdc[21463](info): set up 2 sockets
Aug 10 19:37:27 dist-test-slave-tm1g krb5kdc[21463](info): commencing operation
krb5kdc: starting...
W20260810 19:37:28.330407 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:28 dist-test-slave-tm1g krb5kdc[21463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:28.657965 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:29.004580 21476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:29.004911 21476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:29.005088 21476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:29.012166 21476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:29.012317 21476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:29.012434 21476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:29.012543 21476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:29.012632 21476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:29.022388 21476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:29.023710 21476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:29.025084 21476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:29.037516 21481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:29.037562 21482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:29.037597 21485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:29.038407 21476 server_base.cc:1034] running on GCE node
I20260810 19:37:29.039242 21476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:29.040616 21476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:29.041848 21476 hybrid_clock.cc:648] HybridClock initialized: now 1786390649041818 us; error 45 us; skew 500 ppm
I20260810 19:37:29.042198 21476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:29 dist-test-slave-tm1g krb5kdc[21463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:29.045346 21476 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:29.046936 21476 webserver.cc:460] Webserver started at http://127.0.0.1:35783/ using document root <none> and password file <none>
I20260810 19:37:29.047621 21476 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:29.047772 21476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:29.048082 21476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:29.051105 21476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "89b2e506db2e4041beb4b1fbae606903"
format_stamp: "Formatted at 2026-08-10 19:37:29 on dist-test-slave-tm1g"
server_key: "54d3f8adab797a005ca10eb6f586a20a"
server_key_iv: "4e366d6bdbf1b8529d05d2edeb40c269"
server_key_version: "encryptionkey@0"
I20260810 19:37:29.051877 21476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "89b2e506db2e4041beb4b1fbae606903"
format_stamp: "Formatted at 2026-08-10 19:37:29 on dist-test-slave-tm1g"
server_key: "54d3f8adab797a005ca10eb6f586a20a"
server_key_iv: "4e366d6bdbf1b8529d05d2edeb40c269"
server_key_version: "encryptionkey@0"
I20260810 19:37:29.056964 21476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:37:29.060336 21493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:29.061612 21476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:37:29.061820 21476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "89b2e506db2e4041beb4b1fbae606903"
format_stamp: "Formatted at 2026-08-10 19:37:29 on dist-test-slave-tm1g"
server_key: "54d3f8adab797a005ca10eb6f586a20a"
server_key_iv: "4e366d6bdbf1b8529d05d2edeb40c269"
server_key_version: "encryptionkey@0"
I20260810 19:37:29.062009 21476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:29.080034 21476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:29.084283 21476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:29.084583 21476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:29.103852 21476 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34527
I20260810 19:37:29.103893 21546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34527 every 8 connection(s)
I20260810 19:37:29.105232 21476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:29.109054 21547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:29.109586 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21476
I20260810 19:37:29.109810 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:29.110266 30630 external_mini_cluster.cc:1428] Setting key 7ef9d2878153502a768b249cdfac8820
I20260810 19:37:29.124073 21547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903: Bootstrap starting.
I20260810 19:37:29.125988 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:29.112191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:39957 (local address 127.0.0.1:34527)
0810 19:37:29.113059 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:29.113073 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:29.113129 (+ 56us) server_negotiation.cc:407] Connection header received
0810 19:37:29.115866 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:29.115893 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:29.116414 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:29.116901 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:29.117422 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:29.119333 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:29.120657 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:29.121767 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:29.121933 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:29.123186 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:29.123196 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:29.123213 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:29.123929 (+ 716us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:29.124036 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:29.124077 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:29.124248 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:29.124514 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 19:37:29.124685 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":364,"threads_started":1}
I20260810 19:37:29.127696 21547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:29.128924 21547 log.cc:824] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:29.129771 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:29.132138 21547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903: No bootstrap required, opened a new log
I20260810 19:37:29.141773 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:29.135038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:48605 (local address 127.0.0.1:34527)
0810 19:37:29.135222 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:29.135229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:29.135692 (+ 463us) server_negotiation.cc:407] Connection header received
0810 19:37:29.135818 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:29.135823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:29.135916 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:29.136032 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:29.136490 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:29.137672 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:29.139069 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:29.139773 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:29.139918 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:29.141024 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:29.141031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:29.141036 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:29.141057 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:29.141122 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:29.141142 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:29.141251 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:29.141422 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 19:37:29.141473 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 19:37:29.142144 21547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b2e506db2e4041beb4b1fbae606903" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260810 19:37:29.142639 21547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:29.142761 21547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b2e506db2e4041beb4b1fbae606903, State: Initialized, Role: FOLLOWER
I20260810 19:37:29.143416 21547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b2e506db2e4041beb4b1fbae606903" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260810 19:37:29.143679 21547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:29.143834 21547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:29.144016 21547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:29.146178 21547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b2e506db2e4041beb4b1fbae606903" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260810 19:37:29.146734 21547 leader_election.cc:302] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89b2e506db2e4041beb4b1fbae606903; no voters:
I20260810 19:37:29.147312 21547 leader_election.cc:288] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:29.147421 21557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:29.148146 21557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [term 1 LEADER]: Becoming Leader. State: Replica: 89b2e506db2e4041beb4b1fbae606903, State: Running, Role: LEADER
I20260810 19:37:29.148716 21557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b2e506db2e4041beb4b1fbae606903" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } }
I20260810 19:37:29.149719 21547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:29.151777 21558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89b2e506db2e4041beb4b1fbae606903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b2e506db2e4041beb4b1fbae606903" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } } }
I20260810 19:37:29.152262 21558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:29.152745 21559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89b2e506db2e4041beb4b1fbae606903. Latest consensus state: current_term: 1 leader_uuid: "89b2e506db2e4041beb4b1fbae606903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b2e506db2e4041beb4b1fbae606903" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34527 } } }
I20260810 19:37:29.154699 21559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:29.156181 21567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:29.160553 21567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:29.169373 21567 catalog_manager.cc:1347] Generated new cluster ID: 37b4f4e2935c4f018dba6c215fba90db
I20260810 19:37:29.169488 21567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:29.192270 21567 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:29.193616 21567 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:29.200451 21567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89b2e506db2e4041beb4b1fbae606903: Generated new TSK 0
I20260810 19:37:29.201326 21567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:29.211443 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21476
W20260810 19:37:29.212736 21541 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:39957
W20260810 19:37:29.213518 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21476: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:29.213647 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21476
W20260810 19:37:29.234524 21461 connection.cc:535] client connection to 127.0.0.1:34527 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34527): Connection reset by peer (error 104)
2026-08-10T19:37:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2077 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:29 dist-test-slave-tm1g krb5kdc[21581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:29 dist-test-slave-tm1g krb5kdc[21581](info): set up 2 sockets
Aug 10 19:37:29 dist-test-slave-tm1g krb5kdc[21581](info): commencing operation
krb5kdc: starting...
W20260810 19:37:30.402921 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:30 dist-test-slave-tm1g krb5kdc[21581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:30.727926 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:31.075160 21597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:31.075446 21597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:31.075546 21597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:31.082821 21597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:31.082928 21597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:31.082975 21597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:31.083027 21597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:31.083072 21597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:31.092782 21597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:31.093974 21597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:31.095244 21597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:31.107524 21603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:31.107550 21605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:31.107859 21602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:31.108283 21597 server_base.cc:1034] running on GCE node
I20260810 19:37:31.109031 21597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:31.110453 21597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:31.111696 21597 hybrid_clock.cc:648] HybridClock initialized: now 1786390651111649 us; error 45 us; skew 500 ppm
I20260810 19:37:31.112068 21597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:31 dist-test-slave-tm1g krb5kdc[21581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390651, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:31.115303 21597 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:31.116737 21597 webserver.cc:460] Webserver started at http://127.0.0.1:41003/ using document root <none> and password file <none>
I20260810 19:37:31.117434 21597 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:31.117583 21597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:31.117892 21597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:31.120918 21597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "7c69c1717bd84c0bbe22755e08da81da"
format_stamp: "Formatted at 2026-08-10 19:37:31 on dist-test-slave-tm1g"
server_key: "180cb91830945688f3fed55f9c18cd0e"
server_key_iv: "23919677f4596c96a8a7cd71e90c6646"
server_key_version: "encryptionkey@0"
I20260810 19:37:31.121699 21597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "7c69c1717bd84c0bbe22755e08da81da"
format_stamp: "Formatted at 2026-08-10 19:37:31 on dist-test-slave-tm1g"
server_key: "180cb91830945688f3fed55f9c18cd0e"
server_key_iv: "23919677f4596c96a8a7cd71e90c6646"
server_key_version: "encryptionkey@0"
I20260810 19:37:31.126825 21597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 19:37:31.130539 21612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:31.131891 21597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:31.132072 21597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "7c69c1717bd84c0bbe22755e08da81da"
format_stamp: "Formatted at 2026-08-10 19:37:31 on dist-test-slave-tm1g"
server_key: "180cb91830945688f3fed55f9c18cd0e"
server_key_iv: "23919677f4596c96a8a7cd71e90c6646"
server_key_version: "encryptionkey@0"
I20260810 19:37:31.132282 21597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:31.138101 21597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:31.142072 21597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:31.142436 21597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:31.160279 21597 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34015
I20260810 19:37:31.160360 21665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34015 every 8 connection(s)
I20260810 19:37:31.161643 21597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:31.165299 21666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:31.169765 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21597
I20260810 19:37:31.169953 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:31.170457 30630 external_mini_cluster.cc:1428] Setting key 322693321abe7ca2d9d4ff75b632e724
I20260810 19:37:31.176247 21666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da: Bootstrap starting.
I20260810 19:37:31.178510 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:31.172397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:58603 (local address 127.0.0.1:34015)
0810 19:37:31.173244 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:31.173256 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:31.173298 (+ 42us) server_negotiation.cc:407] Connection header received
0810 19:37:31.175152 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:31.175173 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:31.175484 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:31.175779 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:31.176916 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:31.176922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:31.176935 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:31.176948 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:31.177005 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:31.177024 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:31.177180 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:31.177316 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 19:37:31.177515 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":411,"threads_started":1}
I20260810 19:37:31.179966 21666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:31.181149 21666 log.cc:824] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:31.182055 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:31.183863 21666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da: No bootstrap required, opened a new log
I20260810 19:37:31.189079 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:31.186503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:60463 (local address 127.0.0.1:34015)
0810 19:37:31.186736 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:31.186743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:31.187119 (+ 376us) server_negotiation.cc:407] Connection header received
0810 19:37:31.187270 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:31.187273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:31.187318 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:31.187396 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:31.188338 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:31.188342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:31.188345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:31.188359 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:31.188399 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:31.188414 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:31.188502 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:31.188654 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 19:37:31.188703 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:37:31.193120 21666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c69c1717bd84c0bbe22755e08da81da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34015 } }
I20260810 19:37:31.193537 21666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:31.193655 21666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c69c1717bd84c0bbe22755e08da81da, State: Initialized, Role: FOLLOWER
I20260810 19:37:31.194262 21666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c69c1717bd84c0bbe22755e08da81da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34015 } }
I20260810 19:37:31.194561 21666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:31.194713 21666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:31.194890 21666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:31.197031 21666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c69c1717bd84c0bbe22755e08da81da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34015 } }
I20260810 19:37:31.197516 21666 leader_election.cc:302] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c69c1717bd84c0bbe22755e08da81da; no voters:
I20260810 19:37:31.198173 21666 leader_election.cc:288] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:31.198348 21676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:31.199247 21676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [term 1 LEADER]: Becoming Leader. State: Replica: 7c69c1717bd84c0bbe22755e08da81da, State: Running, Role: LEADER
I20260810 19:37:31.199769 21676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c69c1717bd84c0bbe22755e08da81da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34015 } }
I20260810 19:37:31.200344 21666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:31.202471 21677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c69c1717bd84c0bbe22755e08da81da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c69c1717bd84c0bbe22755e08da81da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34015 } } }
I20260810 19:37:31.202910 21677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:31.203245 21678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c69c1717bd84c0bbe22755e08da81da. Latest consensus state: current_term: 1 leader_uuid: "7c69c1717bd84c0bbe22755e08da81da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c69c1717bd84c0bbe22755e08da81da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34015 } } }
I20260810 19:37:31.204216 21678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:31.214182 21682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:31.219489 21682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:31.228644 21682 catalog_manager.cc:1347] Generated new cluster ID: 93333e2f0ef542bc9f39080151c0a841
I20260810 19:37:31.228762 21682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:31.259557 21682 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:31.260026 21682 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:31.274950 21682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c69c1717bd84c0bbe22755e08da81da: Generated new TSK 0
I20260810 19:37:31.275815 21682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:31.290196 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21597
W20260810 19:37:31.291316 21661 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:58603
W20260810 19:37:31.292047 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21597: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:31.292217 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21597
2026-08-10T19:37:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2078 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:31 dist-test-slave-tm1g krb5kdc[21699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:31 dist-test-slave-tm1g krb5kdc[21699](info): set up 2 sockets
Aug 10 19:37:31 dist-test-slave-tm1g krb5kdc[21699](info): commencing operation
krb5kdc: starting...
W20260810 19:37:32.478932 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:32 dist-test-slave-tm1g krb5kdc[21699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:32.810664 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:33.163002 21713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:33.163286 21713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:33.163457 21713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:33.170785 21713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:33.170890 21713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:33.170998 21713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:33.171085 21713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:33.171168 21713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:33.181293 21713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42135
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:33.182659 21713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:33.184082 21713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:33.196295 21718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:33.196327 21721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:33.196416 21719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:33.197098 21713 server_base.cc:1034] running on GCE node
I20260810 19:37:33.197896 21713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:33.199329 21713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:33.200534 21713 hybrid_clock.cc:648] HybridClock initialized: now 1786390653200493 us; error 29 us; skew 500 ppm
I20260810 19:37:33.200893 21713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:33 dist-test-slave-tm1g krb5kdc[21699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:33.208343 21713 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:33.209806 21713 webserver.cc:460] Webserver started at http://127.0.0.1:37987/ using document root <none> and password file <none>
I20260810 19:37:33.210673 21713 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:33.210813 21713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:33.211145 21713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:33.214265 21713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "de890aaca9c14d408bb963df12edaf26"
format_stamp: "Formatted at 2026-08-10 19:37:33 on dist-test-slave-tm1g"
server_key: "443f257378a2238f9c6c3549fe174ff2"
server_key_iv: "82aadcdbd9b978b2c0ac6c74cac32289"
server_key_version: "encryptionkey@0"
I20260810 19:37:33.215143 21713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "de890aaca9c14d408bb963df12edaf26"
format_stamp: "Formatted at 2026-08-10 19:37:33 on dist-test-slave-tm1g"
server_key: "443f257378a2238f9c6c3549fe174ff2"
server_key_iv: "82aadcdbd9b978b2c0ac6c74cac32289"
server_key_version: "encryptionkey@0"
I20260810 19:37:33.220266 21713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 19:37:33.223697 21729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:33.225073 21713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:33.225277 21713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "de890aaca9c14d408bb963df12edaf26"
format_stamp: "Formatted at 2026-08-10 19:37:33 on dist-test-slave-tm1g"
server_key: "443f257378a2238f9c6c3549fe174ff2"
server_key_iv: "82aadcdbd9b978b2c0ac6c74cac32289"
server_key_version: "encryptionkey@0"
I20260810 19:37:33.225682 21713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:33.239188 21713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:33.243621 21713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:33.243917 21713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:33.263147 21713 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42135
I20260810 19:37:33.263185 21786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42135 every 8 connection(s)
I20260810 19:37:33.264606 21713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:33.268409 21787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:33.274894 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21713
I20260810 19:37:33.275089 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:33.275569 30630 external_mini_cluster.cc:1428] Setting key 6e150f59528809a5b6461f63d43d65d8
I20260810 19:37:33.279572 21787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26: Bootstrap starting.
I20260810 19:37:33.283459 21787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:33.284551 21787 log.cc:824] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:33.287880 21787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26: No bootstrap required, opened a new log
Aug 10 19:37:33 dist-test-slave-tm1g krb5kdc[21699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 19:37:33.296010 21787 raft_consensus.cc:348] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de890aaca9c14d408bb963df12edaf26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260810 19:37:33.296459 21787 raft_consensus.cc:374] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:33.296604 21787 raft_consensus.cc:729] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de890aaca9c14d408bb963df12edaf26, State: Initialized, Role: FOLLOWER
I20260810 19:37:33.297186 21787 consensus_queue.cc:260] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de890aaca9c14d408bb963df12edaf26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260810 19:37:33.297466 21787 raft_consensus.cc:388] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:33.297623 21787 raft_consensus.cc:482] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:33.297804 21787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:33.300405 21787 raft_consensus.cc:504] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de890aaca9c14d408bb963df12edaf26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260810 19:37:33.300779 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:33.277692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:57195 (local address 127.0.0.1:42135)
0810 19:37:33.278527 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:33.278539 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:33.278580 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:37:33.280396 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:33.280419 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:33.280824 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:33.281176 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:33.283168 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:33.284614 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:33.286285 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:33.287019 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:33.287156 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:33.291590 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:33.291602 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:33.291617 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:33.291657 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:33.295396 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:33.296355 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:33.296363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:33.296370 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:33.296689 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:33.297291 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:33.297297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:33.297300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:33.298157 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:33.298578 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:33.299191 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 19:37:33.299579 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":371,"threads_started":1}
I20260810 19:37:33.301016 21787 leader_election.cc:302] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de890aaca9c14d408bb963df12edaf26; no voters:
I20260810 19:37:33.301715 21787 leader_election.cc:288] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:33.301898 21793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:33.304013 21787 sys_catalog.cc:564] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:33.304171 21793 raft_consensus.cc:686] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [term 1 LEADER]: Becoming Leader. State: Replica: de890aaca9c14d408bb963df12edaf26, State: Running, Role: LEADER
I20260810 19:37:33.304927 21793 consensus_queue.cc:237] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de890aaca9c14d408bb963df12edaf26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260810 19:37:33.305650 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:33.308195 21795 sys_catalog.cc:455] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de890aaca9c14d408bb963df12edaf26. Latest consensus state: current_term: 1 leader_uuid: "de890aaca9c14d408bb963df12edaf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de890aaca9c14d408bb963df12edaf26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } } }
I20260810 19:37:33.309185 21795 sys_catalog.cc:458] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:33.308856 21794 sys_catalog.cc:455] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de890aaca9c14d408bb963df12edaf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de890aaca9c14d408bb963df12edaf26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } } }
I20260810 19:37:33.310531 21794 sys_catalog.cc:458] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:33.312301 21805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:33.317890 21805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:33.339039 21805 catalog_manager.cc:1347] Generated new cluster ID: 67ce0be836b0412f9546e00d326e6ee2
I20260810 19:37:33.339176 21805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:33.347512 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:33.323316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:58885 (local address 127.0.0.1:42135)
0810 19:37:33.323552 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:33.323560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:33.323592 (+ 32us) server_negotiation.cc:407] Connection header received
0810 19:37:33.323678 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:33.323686 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:33.323787 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:33.323927 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:33.326390 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:33.328513 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:33.331417 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:33.332567 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:33.332719 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:33.336325 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:33.336334 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:33.336339 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:33.336386 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:33.342981 (+ 6595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:33.344109 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:33.344116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:33.344120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:33.344269 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:33.344831 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:33.344836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:33.344852 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:33.345242 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:33.346708 (+ 1466us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:33.346901 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 19:37:33.347096 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 19:37:33.354085 21805 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:33.355429 21805 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:33.362953 21805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de890aaca9c14d408bb963df12edaf26: Generated new TSK 0
I20260810 19:37:33.363991 21805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:33.376827 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21713
I20260810 19:37:33.378106 21782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:33.471271 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21713
W20260810 19:37:33.492277 21697 connection.cc:535] client connection to 127.0.0.1:42135 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42135): Connection reset by peer (error 104)
2026-08-10T19:37:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2180 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:33 dist-test-slave-tm1g krb5kdc[21822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:33 dist-test-slave-tm1g krb5kdc[21822](info): set up 2 sockets
Aug 10 19:37:33 dist-test-slave-tm1g krb5kdc[21822](info): commencing operation
krb5kdc: starting...
W20260810 19:37:34.661767 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:34 dist-test-slave-tm1g krb5kdc[21822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:34.989770 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:35.341039 21838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:35.341392 21838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:35.341578 21838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:35.349056 21838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:35.349215 21838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:35.349315 21838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:35.349437 21838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:35.349527 21838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:35.359701 21838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:35.361127 21838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:35.362496 21838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:35.374752 21844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:35.374779 21846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:35.375344 21849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:35.375630 21838 server_base.cc:1034] running on GCE node
I20260810 19:37:35.376475 21838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:35.377915 21838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:35.379155 21838 hybrid_clock.cc:648] HybridClock initialized: now 1786390655379100 us; error 50 us; skew 500 ppm
I20260810 19:37:35.379521 21838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:35 dist-test-slave-tm1g krb5kdc[21822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:35.382846 21838 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:35.384398 21838 webserver.cc:460] Webserver started at http://127.0.0.1:37607/ using document root <none> and password file <none>
I20260810 19:37:35.385113 21838 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:35.385264 21838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:35.385594 21838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:35.388649 21838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "8f83328661854299beb8fd6b23e3c8c2"
format_stamp: "Formatted at 2026-08-10 19:37:35 on dist-test-slave-tm1g"
server_key: "856e21f7380800a9509507aaa6481a2a"
server_key_iv: "3c11a1d4a01c12741924ab770dcb58ff"
server_key_version: "encryptionkey@0"
I20260810 19:37:35.389458 21838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "8f83328661854299beb8fd6b23e3c8c2"
format_stamp: "Formatted at 2026-08-10 19:37:35 on dist-test-slave-tm1g"
server_key: "856e21f7380800a9509507aaa6481a2a"
server_key_iv: "3c11a1d4a01c12741924ab770dcb58ff"
server_key_version: "encryptionkey@0"
I20260810 19:37:35.397051 21838 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 19:37:35.400940 21856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:35.402486 21838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 19:37:35.402664 21838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "8f83328661854299beb8fd6b23e3c8c2"
format_stamp: "Formatted at 2026-08-10 19:37:35 on dist-test-slave-tm1g"
server_key: "856e21f7380800a9509507aaa6481a2a"
server_key_iv: "3c11a1d4a01c12741924ab770dcb58ff"
server_key_version: "encryptionkey@0"
I20260810 19:37:35.402850 21838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:35.420933 21838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:35.425616 21838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:35.425937 21838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:35.445822 21838 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33421
I20260810 19:37:35.445878 21913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33421 every 8 connection(s)
I20260810 19:37:35.447280 21838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:35.450337 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21838
I20260810 19:37:35.450513 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:35.451143 30630 external_mini_cluster.cc:1428] Setting key af440bdd12222a837abf2d808c623000
I20260810 19:37:35.451556 21914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:35.465643 21917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:35.453048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:49739 (local address 127.0.0.1:33421)
0810 19:37:35.453839 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:35.453851 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:35.453892 (+ 41us) server_negotiation.cc:407] Connection header received
0810 19:37:35.456328 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:35.456352 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:35.456769 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:35.457110 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:35.457735 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:35.459209 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:35.461069 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:35.461752 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:35.461900 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:35.463260 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:35.463268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:35.463285 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:35.463869 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:35.463953 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:35.463998 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:35.464118 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:35.464398 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 19:37:35.464612 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":357,"threads_started":1}
I20260810 19:37:35.465763 21914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2: Bootstrap starting.
I20260810 19:37:35.469089 21914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:35.469676 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:35.470360 21914 log.cc:824] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:35.473759 21914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2: No bootstrap required, opened a new log
I20260810 19:37:35.483042 21917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:35.474929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:57415 (local address 127.0.0.1:33421)
0810 19:37:35.475121 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:35.475127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:35.475569 (+ 442us) server_negotiation.cc:407] Connection header received
0810 19:37:35.475723 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:35.475729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:35.475817 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:35.475938 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:35.476461 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:35.477674 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:35.479170 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:35.479903 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:35.480069 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:35.481737 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:35.481745 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:35.481752 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:35.481781 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:35.481863 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:35.481887 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:35.482040 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:35.482397 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 19:37:35.482688 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 19:37:35.484437 21914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f83328661854299beb8fd6b23e3c8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260810 19:37:35.484956 21914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:35.485085 21914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f83328661854299beb8fd6b23e3c8c2, State: Initialized, Role: FOLLOWER
I20260810 19:37:35.485682 21914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f83328661854299beb8fd6b23e3c8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260810 19:37:35.485922 21914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:35.486064 21914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:35.486261 21914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:35.488551 21914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f83328661854299beb8fd6b23e3c8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260810 19:37:35.489095 21914 leader_election.cc:302] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f83328661854299beb8fd6b23e3c8c2; no voters:
I20260810 19:37:35.489698 21914 leader_election.cc:288] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:35.489919 21928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:35.491436 21928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [term 1 LEADER]: Becoming Leader. State: Replica: 8f83328661854299beb8fd6b23e3c8c2, State: Running, Role: LEADER
I20260810 19:37:35.492022 21914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:35.491989 21928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f83328661854299beb8fd6b23e3c8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260810 19:37:35.494957 21930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f83328661854299beb8fd6b23e3c8c2. Latest consensus state: current_term: 1 leader_uuid: "8f83328661854299beb8fd6b23e3c8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f83328661854299beb8fd6b23e3c8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } } }
I20260810 19:37:35.495424 21930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:35.495034 21929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f83328661854299beb8fd6b23e3c8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f83328661854299beb8fd6b23e3c8c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } } }
I20260810 19:37:35.495814 21929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:35.498684 21937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:35.504014 21937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:35.513001 21937 catalog_manager.cc:1347] Generated new cluster ID: d71b94f6758e44da886f28dcf58a048f
I20260810 19:37:35.513123 21937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:35.534238 21937 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:35.536242 21937 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:35.550661 21937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f83328661854299beb8fd6b23e3c8c2: Generated new TSK 0
I20260810 19:37:35.551801 21937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:35.569706 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21838
W20260810 19:37:35.571125 21909 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:49739
W20260810 19:37:35.572021 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21838: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:35.572165 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21838
W20260810 19:37:35.590440 21820 connection.cc:535] client connection to 127.0.0.1:33421 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33421): Connection reset by peer (error 104)
2026-08-10T19:37:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2097 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:35 dist-test-slave-tm1g krb5kdc[21950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:35 dist-test-slave-tm1g krb5kdc[21950](info): set up 2 sockets
Aug 10 19:37:35 dist-test-slave-tm1g krb5kdc[21950](info): commencing operation
krb5kdc: starting...
W20260810 19:37:36.757265 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:36 dist-test-slave-tm1g krb5kdc[21950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 19:37:37.095703 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:37.447599 21966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:37.447906 21966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:37.448087 21966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:37.455380 21966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:37.455503 21966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:37.455610 21966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:37.455720 21966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:37.455806 21966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:37.465845 21966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35887
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:37.467274 21966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:37.468694 21966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:37.481117 21972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:37.481339 21973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:37.481117 21975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:37.481989 21966 server_base.cc:1034] running on GCE node
I20260810 19:37:37.482930 21966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:37.484429 21966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:37.485692 21966 hybrid_clock.cc:648] HybridClock initialized: now 1786390657485649 us; error 48 us; skew 500 ppm
I20260810 19:37:37.486074 21966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:37 dist-test-slave-tm1g krb5kdc[21950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:37.490479 21966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 19:37:37.492054 21966 webserver.cc:460] Webserver started at http://127.0.0.1:42203/ using document root <none> and password file <none>
I20260810 19:37:37.492810 21966 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:37.492969 21966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:37.493309 21966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:37.496554 21966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "28da04fbbf0444638d7ec283abd03403"
format_stamp: "Formatted at 2026-08-10 19:37:37 on dist-test-slave-tm1g"
server_key: "aa806b2fbd3e8ec6f414f0338396b751"
server_key_iv: "2b4c4ae1f9c6226a450851783dae0276"
server_key_version: "encryptionkey@0"
I20260810 19:37:37.497392 21966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "28da04fbbf0444638d7ec283abd03403"
format_stamp: "Formatted at 2026-08-10 19:37:37 on dist-test-slave-tm1g"
server_key: "aa806b2fbd3e8ec6f414f0338396b751"
server_key_iv: "2b4c4ae1f9c6226a450851783dae0276"
server_key_version: "encryptionkey@0"
I20260810 19:37:37.502964 21966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 19:37:37.506600 21986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:37.507973 21966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:37.508163 21966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "28da04fbbf0444638d7ec283abd03403"
format_stamp: "Formatted at 2026-08-10 19:37:37 on dist-test-slave-tm1g"
server_key: "aa806b2fbd3e8ec6f414f0338396b751"
server_key_iv: "2b4c4ae1f9c6226a450851783dae0276"
server_key_version: "encryptionkey@0"
I20260810 19:37:37.508375 21966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:37.514309 21966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:37.518815 21966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:37.519129 21966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:37.539067 21966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35887
I20260810 19:37:37.539117 22042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35887 every 8 connection(s)
I20260810 19:37:37.540601 21966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:37.544484 22043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:37.549826 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 21966
I20260810 19:37:37.550009 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:37.550570 30630 external_mini_cluster.cc:1428] Setting key 80aa41059714a4ecde3eda19a9bc9d7b
I20260810 19:37:37.556246 22043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403: Bootstrap starting.
I20260810 19:37:37.558861 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:37.552497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:59069 (local address 127.0.0.1:35887)
0810 19:37:37.553534 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:37.553546 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:37.553595 (+ 49us) server_negotiation.cc:407] Connection header received
0810 19:37:37.555366 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:37.555390 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:37.555756 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:37.556104 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:37.557164 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:37.557170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:37.557183 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:37.557198 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:37.557256 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:37.557275 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:37.557442 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:37.557608 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 19:37:37.557819 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":480,"threads_started":1}
I20260810 19:37:37.559870 22043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:37.561036 22043 log.cc:824] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:37.562624 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:37.563973 22043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403: No bootstrap required, opened a new log
I20260810 19:37:37.569684 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:37.567155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:52111 (local address 127.0.0.1:35887)
0810 19:37:37.567380 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:37.567392 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:37.567736 (+ 344us) server_negotiation.cc:407] Connection header received
0810 19:37:37.567898 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:37.567902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:37.567947 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:37.568026 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:37.568988 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:37.568992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:37.568996 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:37.569014 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:37.569063 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:37.569080 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:37.569170 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:37.569315 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 19:37:37.569364 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 19:37:37.572784 22043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28da04fbbf0444638d7ec283abd03403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260810 19:37:37.573174 22043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:37.573283 22043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28da04fbbf0444638d7ec283abd03403, State: Initialized, Role: FOLLOWER
I20260810 19:37:37.573899 22043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28da04fbbf0444638d7ec283abd03403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260810 19:37:37.574168 22043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:37.574326 22043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:37.574520 22043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:37.576706 22043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28da04fbbf0444638d7ec283abd03403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260810 19:37:37.577204 22043 leader_election.cc:302] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28da04fbbf0444638d7ec283abd03403; no voters:
I20260810 19:37:37.577787 22043 leader_election.cc:288] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:37.577920 22056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:37.578675 22056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [term 1 LEADER]: Becoming Leader. State: Replica: 28da04fbbf0444638d7ec283abd03403, State: Running, Role: LEADER
I20260810 19:37:37.579195 22056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28da04fbbf0444638d7ec283abd03403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260810 19:37:37.579931 22043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:37.582432 22058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28da04fbbf0444638d7ec283abd03403. Latest consensus state: current_term: 1 leader_uuid: "28da04fbbf0444638d7ec283abd03403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28da04fbbf0444638d7ec283abd03403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } } }
I20260810 19:37:37.582849 22058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:37.583338 22057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28da04fbbf0444638d7ec283abd03403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28da04fbbf0444638d7ec283abd03403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } } }
I20260810 19:37:37.583801 22057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:37.587347 22063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:37.592892 22063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:37.602254 22063 catalog_manager.cc:1347] Generated new cluster ID: 4d967a7549614309ba2f0e507538d306
I20260810 19:37:37.602406 22063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:37.615864 22063 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:37.616362 22063 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:37.636741 22063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28da04fbbf0444638d7ec283abd03403: Generated new TSK 0
I20260810 19:37:37.637943 22063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:37.652454 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21966
W20260810 19:37:37.653658 22037 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.78:59069
W20260810 19:37:37.654418 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 21966: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:37.654551 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 21966
2026-08-10T19:37:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:37 dist-test-slave-tm1g krb5kdc[22083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:37 dist-test-slave-tm1g krb5kdc[22083](info): set up 2 sockets
Aug 10 19:37:37 dist-test-slave-tm1g krb5kdc[22083](info): commencing operation
krb5kdc: starting...
W20260810 19:37:38.854283 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:39 dist-test-slave-tm1g krb5kdc[22083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:39.197446 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:44541
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:37355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:44541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:39.542107 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:39.542464 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:39.542567 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:39.549860 22097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:39.549966 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:39.550020 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:39.550084 22097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:39.550127 22097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:39.559998 22097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:37355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:44541
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:44541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:39.561296 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:39.562633 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:39.574620 22103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:39.574692 22104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:39.574694 22106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:39.575891 22097 server_base.cc:1034] running on GCE node
I20260810 19:37:39.576700 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:39.578105 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:39.579375 22097 hybrid_clock.cc:648] HybridClock initialized: now 1786390659579333 us; error 57 us; skew 500 ppm
I20260810 19:37:39.579742 22097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:39 dist-test-slave-tm1g krb5kdc[22083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390659, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:39.587761 22097 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:39.589452 22097 webserver.cc:460] Webserver started at http://127.29.233.190:44975/ using document root <none> and password file <none>
I20260810 19:37:39.590211 22097 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:39.590401 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:39.590734 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:39.593917 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "473fa24119174534b3b01f2eeb62500c"
format_stamp: "Formatted at 2026-08-10 19:37:39 on dist-test-slave-tm1g"
server_key: "522ee23aa6fea7228d12deeb575fe869"
server_key_iv: "afba2999a653b91aa097c1981fc0c163"
server_key_version: "encryptionkey@0"
I20260810 19:37:39.594749 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "473fa24119174534b3b01f2eeb62500c"
format_stamp: "Formatted at 2026-08-10 19:37:39 on dist-test-slave-tm1g"
server_key: "522ee23aa6fea7228d12deeb575fe869"
server_key_iv: "afba2999a653b91aa097c1981fc0c163"
server_key_version: "encryptionkey@0"
I20260810 19:37:39.600064 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 19:37:39.603477 22113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:39.604827 22097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:39.605022 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "473fa24119174534b3b01f2eeb62500c"
format_stamp: "Formatted at 2026-08-10 19:37:39 on dist-test-slave-tm1g"
server_key: "522ee23aa6fea7228d12deeb575fe869"
server_key_iv: "afba2999a653b91aa097c1981fc0c163"
server_key_version: "encryptionkey@0"
I20260810 19:37:39.605225 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:39.649169 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:39.653551 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:39.653877 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:39.673975 22097 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:44541
I20260810 19:37:39.674009 22167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:44541 every 8 connection(s)
I20260810 19:37:39.675424 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:39.679322 22168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:39.679410 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22097
I20260810 19:37:39.679587 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:39.680078 30630 external_mini_cluster.cc:1428] Setting key 7804c8108cd48d08a738f4c17d75c243
I20260810 19:37:39.694258 22168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c: Bootstrap starting.
Aug 10 19:37:39 dist-test-slave-tm1g krb5kdc[22083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:39.697597 22168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:39.698937 22168 log.cc:824] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:39.701749 22168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c: No bootstrap required, opened a new log
I20260810 19:37:39.704026 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:39.681841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45567 (local address 127.29.233.190:44541)
0810 19:37:39.682754 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:39.682767 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:39.682809 (+ 42us) server_negotiation.cc:407] Connection header received
0810 19:37:39.684348 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:39.684369 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:39.684748 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:39.685096 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:39.687079 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:39.688442 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:39.690266 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:39.691063 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:39.691210 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:39.696322 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:39.696330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:39.696343 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:39.696381 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:39.699788 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:39.700600 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:39.700606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:39.700612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:39.700771 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:39.701269 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:39.701273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:39.701275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:39.701774 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:39.702140 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:39.702660 (+ 520us) server_negotiation.cc:299] Negotiation successful
0810 19:37:39.703006 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":452,"threads_started":1}
I20260810 19:37:39.708009 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:39.710181 22168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fa24119174534b3b01f2eeb62500c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44541 } }
I20260810 19:37:39.710603 22168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:39.710719 22168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 473fa24119174534b3b01f2eeb62500c, State: Initialized, Role: FOLLOWER
I20260810 19:37:39.711351 22168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fa24119174534b3b01f2eeb62500c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44541 } }
I20260810 19:37:39.711608 22168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:39.711761 22168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:39.711941 22168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:39.714640 22168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fa24119174534b3b01f2eeb62500c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44541 } }
I20260810 19:37:39.715157 22168 leader_election.cc:302] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 473fa24119174534b3b01f2eeb62500c; no voters:
I20260810 19:37:39.715794 22168 leader_election.cc:288] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:39.715956 22178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:39.717213 22178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [term 1 LEADER]: Becoming Leader. State: Replica: 473fa24119174534b3b01f2eeb62500c, State: Running, Role: LEADER
I20260810 19:37:39.717857 22178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fa24119174534b3b01f2eeb62500c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44541 } }
I20260810 19:37:39.718241 22168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:39.727556 22180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473fa24119174534b3b01f2eeb62500c. Latest consensus state: current_term: 1 leader_uuid: "473fa24119174534b3b01f2eeb62500c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fa24119174534b3b01f2eeb62500c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44541 } } }
I20260810 19:37:39.727973 22180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:39.728765 22179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "473fa24119174534b3b01f2eeb62500c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473fa24119174534b3b01f2eeb62500c" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 44541 } } }
I20260810 19:37:39.729079 22179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:39.730973 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:39.712768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51571 (local address 127.29.233.190:44541)
0810 19:37:39.712960 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:39.712965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:39.713537 (+ 572us) server_negotiation.cc:407] Connection header received
0810 19:37:39.713650 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:39.713656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:39.713739 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:39.713859 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:39.715748 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:39.717174 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:39.718760 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:39.719252 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:39.719394 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:39.721953 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:39.721958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:39.721961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:39.721990 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:39.725064 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:39.725821 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:39.725826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:39.725830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:39.726004 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:39.726672 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:39.726677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:39.726680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:39.727151 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:39.727503 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:39.730469 (+ 2966us) server_negotiation.cc:299] Negotiation successful
0810 19:37:39.730700 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:37:39.733565 22185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:39.739058 22185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:39.752360 22185 catalog_manager.cc:1347] Generated new cluster ID: a8bdd31c881f47c7bd0ceec1b65dab0c
I20260810 19:37:39.752492 22185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:39.790663 22185 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:39.792039 22185 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:39.800870 22185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 473fa24119174534b3b01f2eeb62500c: Generated new TSK 0
I20260810 19:37:39.801924 22185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:39.818034 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22097
I20260810 19:37:39.819285 22163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:39.915189 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22097
W20260810 19:37:39.936576 22081 connection.cc:535] client connection to 127.29.233.190:44541 recv error: Network error: failed to read from TLS socket (remote: 127.29.233.190:44541): Connection reset by peer (error 104)
2026-08-10T19:37:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:40 dist-test-slave-tm1g krb5kdc[22204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:40 dist-test-slave-tm1g krb5kdc[22204](info): set up 2 sockets
Aug 10 19:37:40 dist-test-slave-tm1g krb5kdc[22204](info): commencing operation
krb5kdc: starting...
W20260810 19:37:41.111514 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:41 dist-test-slave-tm1g krb5kdc[22204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:41.451825 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:38809
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:38809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:41.811832 22219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:41.812124 22219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:41.812217 22219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:41.819464 22219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:41.819572 22219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:41.819619 22219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:41.819681 22219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:41.819723 22219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:41.829572 22219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:38809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:38809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:41.830889 22219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:41.832309 22219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:41.844668 22226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:41.844708 22229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:41.844779 22227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:41.845842 22219 server_base.cc:1034] running on GCE node
I20260810 19:37:41.846736 22219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:41.848227 22219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:41.849448 22219 hybrid_clock.cc:648] HybridClock initialized: now 1786390661849397 us; error 42 us; skew 500 ppm
I20260810 19:37:41.849825 22219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:41 dist-test-slave-tm1g krb5kdc[22204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390661, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:41.853462 22219 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:41.855090 22219 webserver.cc:460] Webserver started at http://127.29.233.190:43577/ using document root <none> and password file <none>
I20260810 19:37:41.855818 22219 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:41.855980 22219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:41.856317 22219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:41.859555 22219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "46ad7c4263cc4695853fe8b72e362cbd"
format_stamp: "Formatted at 2026-08-10 19:37:41 on dist-test-slave-tm1g"
server_key: "915a2cc1f814e45d2a8fd8fb591fa9cb"
server_key_iv: "4b6e9e4b8c5863e90ed672050fe9193a"
server_key_version: "encryptionkey@0"
I20260810 19:37:41.860334 22219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "46ad7c4263cc4695853fe8b72e362cbd"
format_stamp: "Formatted at 2026-08-10 19:37:41 on dist-test-slave-tm1g"
server_key: "915a2cc1f814e45d2a8fd8fb591fa9cb"
server_key_iv: "4b6e9e4b8c5863e90ed672050fe9193a"
server_key_version: "encryptionkey@0"
I20260810 19:37:41.865716 22219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 19:37:41.869213 22237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:41.870584 22219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:37:41.870769 22219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "46ad7c4263cc4695853fe8b72e362cbd"
format_stamp: "Formatted at 2026-08-10 19:37:41 on dist-test-slave-tm1g"
server_key: "915a2cc1f814e45d2a8fd8fb591fa9cb"
server_key_iv: "4b6e9e4b8c5863e90ed672050fe9193a"
server_key_version: "encryptionkey@0"
I20260810 19:37:41.870978 22219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:41.898170 22219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:41.902333 22219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:41.902634 22219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:41.921828 22219 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:38809
I20260810 19:37:41.921854 22293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:38809 every 8 connection(s)
I20260810 19:37:41.923349 22219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:41.926399 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22219
I20260810 19:37:41.926601 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:41.927073 30630 external_mini_cluster.cc:1428] Setting key bb7006ebd23ece7700a5f2d1733583e1
I20260810 19:37:41.927246 22294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:41.941843 22297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:41.928981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47063 (local address 127.29.233.190:38809)
0810 19:37:41.929853 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:41.929867 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:41.929927 (+ 60us) server_negotiation.cc:407] Connection header received
0810 19:37:41.932390 (+ 2463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:41.932424 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:41.932852 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:41.933188 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:41.933820 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:41.935286 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:41.937202 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:41.937907 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:41.938049 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:41.939449 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:41.939457 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:41.939470 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:41.940075 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:41.940151 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:41.940176 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:41.940303 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:41.940666 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 19:37:41.940870 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":340,"threads_started":1}
I20260810 19:37:41.942101 22294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd: Bootstrap starting.
I20260810 19:37:41.945360 22294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:41.945811 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:41.946674 22294 log.cc:824] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:41.950076 22294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd: No bootstrap required, opened a new log
I20260810 19:37:41.958495 22297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:41.950995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43277 (local address 127.29.233.190:38809)
0810 19:37:41.951178 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:41.951184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:41.951661 (+ 477us) server_negotiation.cc:407] Connection header received
0810 19:37:41.951809 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:41.951819 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:41.951911 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:41.952023 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:41.952512 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:41.953699 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:41.955553 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:41.956275 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:41.956420 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:41.957597 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:41.957609 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:41.957614 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:41.957635 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:41.957685 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:41.957704 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:41.957830 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:41.958045 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 19:37:41.958101 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:37:41.959898 22294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad7c4263cc4695853fe8b72e362cbd" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38809 } }
I20260810 19:37:41.960374 22294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:41.960479 22294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ad7c4263cc4695853fe8b72e362cbd, State: Initialized, Role: FOLLOWER
I20260810 19:37:41.961071 22294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad7c4263cc4695853fe8b72e362cbd" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38809 } }
I20260810 19:37:41.961308 22294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:41.961472 22294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:41.961640 22294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:41.963999 22294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad7c4263cc4695853fe8b72e362cbd" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38809 } }
I20260810 19:37:41.964507 22294 leader_election.cc:302] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ad7c4263cc4695853fe8b72e362cbd; no voters:
I20260810 19:37:41.965101 22294 leader_election.cc:288] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:41.965268 22304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:41.966634 22304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [term 1 LEADER]: Becoming Leader. State: Replica: 46ad7c4263cc4695853fe8b72e362cbd, State: Running, Role: LEADER
I20260810 19:37:41.967198 22304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad7c4263cc4695853fe8b72e362cbd" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38809 } }
I20260810 19:37:41.967497 22294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:41.970237 22306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ad7c4263cc4695853fe8b72e362cbd. Latest consensus state: current_term: 1 leader_uuid: "46ad7c4263cc4695853fe8b72e362cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad7c4263cc4695853fe8b72e362cbd" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38809 } } }
I20260810 19:37:41.970170 22305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46ad7c4263cc4695853fe8b72e362cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ad7c4263cc4695853fe8b72e362cbd" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 38809 } } }
I20260810 19:37:41.970672 22306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:41.970702 22305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:41.976745 22312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:41.982090 22312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:41.991433 22312 catalog_manager.cc:1347] Generated new cluster ID: 9e6c4605015343bb8f38ed71aa7d0ac9
I20260810 19:37:41.991547 22312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:42.011126 22312 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:42.012529 22312 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:42.018716 22312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ad7c4263cc4695853fe8b72e362cbd: Generated new TSK 0
I20260810 19:37:42.019567 22312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:42.029163 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22219
W20260810 19:37:42.030470 22289 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47063
W20260810 19:37:42.031253 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22219: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:42.031409 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22219
W20260810 19:37:42.052445 22202 connection.cc:535] client connection to 127.29.233.190:38809 recv error: Network error: failed to read from TLS socket (remote: 127.29.233.190:38809): Connection reset by peer (error 104)
2026-08-10T19:37:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2115 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:42 dist-test-slave-tm1g krb5kdc[22327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:42 dist-test-slave-tm1g krb5kdc[22327](info): set up 2 sockets
Aug 10 19:37:42 dist-test-slave-tm1g krb5kdc[22327](info): commencing operation
krb5kdc: starting...
W20260810 19:37:43.222733 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:43 dist-test-slave-tm1g krb5kdc[22327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:43.562145 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:40145
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:42381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:40145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:43.914657 22344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:43.914951 22344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:43.915153 22344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:43.922478 22344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:43.922585 22344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:43.922658 22344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:43.922775 22344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:43.922837 22344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:43.932744 22344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:42381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:40145
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:40145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:43.934119 22344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:43.935530 22344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:43.948063 22351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:43.948135 22350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:43.948176 22353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:43.948909 22344 server_base.cc:1034] running on GCE node
I20260810 19:37:43.949626 22344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:43.951028 22344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:43.952240 22344 hybrid_clock.cc:648] HybridClock initialized: now 1786390663952193 us; error 37 us; skew 500 ppm
I20260810 19:37:43.952586 22344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:43 dist-test-slave-tm1g krb5kdc[22327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390663, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:43.955816 22344 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:43.957231 22344 webserver.cc:460] Webserver started at http://127.29.233.190:37295/ using document root <none> and password file <none>
I20260810 19:37:43.957906 22344 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:43.958036 22344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:43.958415 22344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:43.961334 22344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "a7f590ea380e486ba665907a473da1f0"
format_stamp: "Formatted at 2026-08-10 19:37:43 on dist-test-slave-tm1g"
server_key: "9eb5ad374944f6f6364f88601c69eb17"
server_key_iv: "56aa2df0a4d8260f1b1dff9a6545119f"
server_key_version: "encryptionkey@0"
I20260810 19:37:43.962091 22344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "a7f590ea380e486ba665907a473da1f0"
format_stamp: "Formatted at 2026-08-10 19:37:43 on dist-test-slave-tm1g"
server_key: "9eb5ad374944f6f6364f88601c69eb17"
server_key_iv: "56aa2df0a4d8260f1b1dff9a6545119f"
server_key_version: "encryptionkey@0"
I20260810 19:37:43.967384 22344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 19:37:43.970835 22362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:43.972246 22344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 19:37:43.972447 22344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "a7f590ea380e486ba665907a473da1f0"
format_stamp: "Formatted at 2026-08-10 19:37:43 on dist-test-slave-tm1g"
server_key: "9eb5ad374944f6f6364f88601c69eb17"
server_key_iv: "56aa2df0a4d8260f1b1dff9a6545119f"
server_key_version: "encryptionkey@0"
I20260810 19:37:43.972673 22344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:43.978533 22344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:43.982429 22344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:43.982726 22344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:44.002194 22344 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:40145
I20260810 19:37:44.002238 22418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:40145 every 8 connection(s)
I20260810 19:37:44.003696 22344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:44.004328 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22344
I20260810 19:37:44.004546 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:44.005144 30630 external_mini_cluster.cc:1428] Setting key b49f871d636edcdc1c65a24a3643c13d
I20260810 19:37:44.008224 22419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:44.012907 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:44.006994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50171 (local address 127.29.233.190:40145)
0810 19:37:44.007811 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:44.007822 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:44.007864 (+ 42us) server_negotiation.cc:407] Connection header received
0810 19:37:44.009581 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:44.009602 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:44.009912 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:44.010230 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:44.011366 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:44.011372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:44.011384 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:44.011398 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:44.011489 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:44.011509 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:44.011684 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:44.011782 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 19:37:44.011967 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":378,"threads_started":1}
I20260810 19:37:44.016675 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:44.021132 22419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0: Bootstrap starting.
I20260810 19:37:44.024641 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:44.022022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44309 (local address 127.29.233.190:40145)
0810 19:37:44.022200 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:44.022205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:44.022743 (+ 538us) server_negotiation.cc:407] Connection header received
0810 19:37:44.022862 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:44.022870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:44.022945 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:44.023050 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:44.023990 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:44.023995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:44.023999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:44.024018 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:44.024067 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:44.024089 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:44.024209 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:44.024284 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 19:37:44.024341 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:37:44.024946 22419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:44.026279 22419 log.cc:824] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:44.028942 22419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0: No bootstrap required, opened a new log
I20260810 19:37:44.036401 22419 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f590ea380e486ba665907a473da1f0" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40145 } }
I20260810 19:37:44.036769 22419 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:44.036890 22419 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f590ea380e486ba665907a473da1f0, State: Initialized, Role: FOLLOWER
I20260810 19:37:44.037529 22419 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f590ea380e486ba665907a473da1f0" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40145 } }
I20260810 19:37:44.037768 22419 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:44.037935 22419 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:44.038113 22419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:44.040489 22419 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f590ea380e486ba665907a473da1f0" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40145 } }
I20260810 19:37:44.041003 22419 leader_election.cc:302] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7f590ea380e486ba665907a473da1f0; no voters:
I20260810 19:37:44.041595 22419 leader_election.cc:288] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:44.041743 22430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:44.042465 22430 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [term 1 LEADER]: Becoming Leader. State: Replica: a7f590ea380e486ba665907a473da1f0, State: Running, Role: LEADER
I20260810 19:37:44.042976 22430 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f590ea380e486ba665907a473da1f0" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40145 } }
I20260810 19:37:44.043669 22419 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:44.046123 22432 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f590ea380e486ba665907a473da1f0. Latest consensus state: current_term: 1 leader_uuid: "a7f590ea380e486ba665907a473da1f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f590ea380e486ba665907a473da1f0" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40145 } } }
I20260810 19:37:44.046540 22432 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:44.046667 22431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7f590ea380e486ba665907a473da1f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f590ea380e486ba665907a473da1f0" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 40145 } } }
I20260810 19:37:44.046962 22431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:44.049409 22436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:44.054901 22436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:44.065179 22436 catalog_manager.cc:1347] Generated new cluster ID: 4c5cff847a95438c8087c46a4f4644b5
I20260810 19:37:44.065325 22436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:44.077912 22436 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:44.078402 22436 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:44.092693 22436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7f590ea380e486ba665907a473da1f0: Generated new TSK 0
I20260810 19:37:44.093766 22436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:44.102444 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22344
W20260810 19:37:44.103845 22414 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50171
W20260810 19:37:44.104599 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22344: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:44.104748 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22344
2026-08-10T19:37:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2073 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:44 dist-test-slave-tm1g krb5kdc[22452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:44 dist-test-slave-tm1g krb5kdc[22452](info): set up 2 sockets
Aug 10 19:37:44 dist-test-slave-tm1g krb5kdc[22452](info): commencing operation
krb5kdc: starting...
W20260810 19:37:45.303002 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:45 dist-test-slave-tm1g krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:45.658185 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43623
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:43623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:46.015542 22467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:46.015830 22467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:46.015987 22467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:46.023310 22467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:46.023445 22467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:46.023543 22467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:46.023658 22467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:46.023738 22467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:46.033607 22467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:43623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:43623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:46.034996 22467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:46.036366 22467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:46.048745 22473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:46.048694 22476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:46.048767 22474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:46.049644 22467 server_base.cc:1034] running on GCE node
I20260810 19:37:46.050582 22467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:46.051975 22467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:46.053203 22467 hybrid_clock.cc:648] HybridClock initialized: now 1786390666053127 us; error 58 us; skew 500 ppm
I20260810 19:37:46.053610 22467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:46 dist-test-slave-tm1g krb5kdc[22452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390666, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:46.061131 22467 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:46.062770 22467 webserver.cc:460] Webserver started at http://127.29.233.190:37969/ using document root <none> and password file <none>
I20260810 19:37:46.063450 22467 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:46.063555 22467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:46.063812 22467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:46.066854 22467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "5ceabe1539444333a42efc4e4a966be4"
format_stamp: "Formatted at 2026-08-10 19:37:46 on dist-test-slave-tm1g"
server_key: "c9af8231defc784fd82f0be401f3dbab"
server_key_iv: "4a46f05bea504e35c4490e28d4b8e82e"
server_key_version: "encryptionkey@0"
I20260810 19:37:46.067589 22467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "5ceabe1539444333a42efc4e4a966be4"
format_stamp: "Formatted at 2026-08-10 19:37:46 on dist-test-slave-tm1g"
server_key: "c9af8231defc784fd82f0be401f3dbab"
server_key_iv: "4a46f05bea504e35c4490e28d4b8e82e"
server_key_version: "encryptionkey@0"
I20260810 19:37:46.073051 22467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:37:46.076602 22484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:46.078001 22467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:37:46.078152 22467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "5ceabe1539444333a42efc4e4a966be4"
format_stamp: "Formatted at 2026-08-10 19:37:46 on dist-test-slave-tm1g"
server_key: "c9af8231defc784fd82f0be401f3dbab"
server_key_iv: "4a46f05bea504e35c4490e28d4b8e82e"
server_key_version: "encryptionkey@0"
I20260810 19:37:46.078425 22467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:46.109352 22467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:46.113809 22467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:46.114065 22467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:46.133045 22467 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:43623
I20260810 19:37:46.133069 22536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:43623 every 8 connection(s)
I20260810 19:37:46.134464 22467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:46.138229 22537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:46.144050 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22467
I20260810 19:37:46.144243 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:46.144685 30630 external_mini_cluster.cc:1428] Setting key e385a81bf4d65265f20521ce2bd9f181
I20260810 19:37:46.150393 22537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4: Bootstrap starting.
I20260810 19:37:46.154134 22537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:46.155313 22537 log.cc:824] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:46.158464 22537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4: No bootstrap required, opened a new log
Aug 10 19:37:46 dist-test-slave-tm1g krb5kdc[22452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:46.168550 22537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceabe1539444333a42efc4e4a966be4" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43623 } }
I20260810 19:37:46.169039 22537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:46.169162 22537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ceabe1539444333a42efc4e4a966be4, State: Initialized, Role: FOLLOWER
I20260810 19:37:46.169865 22537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceabe1539444333a42efc4e4a966be4" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43623 } }
I20260810 19:37:46.170079 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:46.146637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35415 (local address 127.29.233.190:43623)
0810 19:37:46.147561 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:46.147577 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:46.147637 (+ 60us) server_negotiation.cc:407] Connection header received
0810 19:37:46.150060 (+ 2423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:46.150086 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:46.150623 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:46.151067 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:46.152407 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:46.154493 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:46.155954 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:46.156769 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:46.156959 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:46.161142 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:46.161152 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:46.161172 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:46.161217 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:46.165361 (+ 4144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:46.166018 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:46.166027 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:46.166040 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:46.166259 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:46.166675 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:46.166680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:46.166683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:46.167454 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:46.167756 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:46.168144 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 19:37:46.168582 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":386,"threads_started":1}
I20260810 19:37:46.170188 22537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:46.170370 22537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:46.170605 22537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:46.173162 22537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceabe1539444333a42efc4e4a966be4" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43623 } }
I20260810 19:37:46.173666 22537 leader_election.cc:302] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ceabe1539444333a42efc4e4a966be4; no voters:
I20260810 19:37:46.173967 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:46.174439 22537 leader_election.cc:288] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:46.174532 22543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:46.176419 22537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:46.180734 22543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [term 1 LEADER]: Becoming Leader. State: Replica: 5ceabe1539444333a42efc4e4a966be4, State: Running, Role: LEADER
I20260810 19:37:46.181358 22543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceabe1539444333a42efc4e4a966be4" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43623 } }
I20260810 19:37:46.184703 22545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ceabe1539444333a42efc4e4a966be4. Latest consensus state: current_term: 1 leader_uuid: "5ceabe1539444333a42efc4e4a966be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceabe1539444333a42efc4e4a966be4" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43623 } } }
I20260810 19:37:46.184715 22544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ceabe1539444333a42efc4e4a966be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceabe1539444333a42efc4e4a966be4" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 43623 } } }
I20260810 19:37:46.185189 22545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:46.185230 22544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:46.187673 22557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:46.193171 22557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:46.203413 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:46.181715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53081 (local address 127.29.233.190:43623)
0810 19:37:46.181949 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:46.181957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:46.182007 (+ 50us) server_negotiation.cc:407] Connection header received
0810 19:37:46.182073 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:46.182081 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:46.182177 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:46.182291 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:46.188117 (+ 5826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:46.189285 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:46.191219 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:46.191703 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:46.191862 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:46.195302 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:46.195312 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:46.195320 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:46.195371 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:46.200359 (+ 4988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:46.201290 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:46.201296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:46.201299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:46.201432 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:46.201817 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:46.201821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:46.201825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:46.202151 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:46.202428 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:46.202799 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 19:37:46.203029 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 19:37:46.208606 22557 catalog_manager.cc:1347] Generated new cluster ID: 09eec3efecf6422a8595cb356c9ccd27
I20260810 19:37:46.208716 22557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:46.235517 22557 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:46.236913 22557 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:46.249585 22557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ceabe1539444333a42efc4e4a966be4: Generated new TSK 0
I20260810 19:37:46.250481 22557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:46.257826 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22467
I20260810 19:37:46.259029 22532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:46.351410 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22467
2026-08-10T19:37:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:46 dist-test-slave-tm1g krb5kdc[22573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:46 dist-test-slave-tm1g krb5kdc[22573](info): set up 2 sockets
Aug 10 19:37:46 dist-test-slave-tm1g krb5kdc[22573](info): commencing operation
krb5kdc: starting...
W20260810 19:37:47.544839 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:47 dist-test-slave-tm1g krb5kdc[22573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:47.901877 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:41097
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:43973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:41097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:48.263903 22586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:48.264256 22586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:48.264431 22586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:48.271565 22586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:48.271721 22586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:48.271834 22586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:48.271967 22586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:48.272079 22586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:48.281757 22586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:41097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:41097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:48.283174 22586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:48.284536 22586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:48.296649 22593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:48.296756 22592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:48.296757 22595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:48.297933 22586 server_base.cc:1034] running on GCE node
I20260810 19:37:48.298867 22586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:48.300331 22586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:48.301571 22586 hybrid_clock.cc:648] HybridClock initialized: now 1786390668301538 us; error 51 us; skew 500 ppm
I20260810 19:37:48.301926 22586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:48 dist-test-slave-tm1g krb5kdc[22573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390668, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:48.305161 22586 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:48.306734 22586 webserver.cc:460] Webserver started at http://127.29.233.190:37077/ using document root <none> and password file <none>
I20260810 19:37:48.307454 22586 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:48.307605 22586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:48.307930 22586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:48.311018 22586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "f244abf270d143de8fdbad15cb4421ed"
format_stamp: "Formatted at 2026-08-10 19:37:48 on dist-test-slave-tm1g"
server_key: "e9c369ca2908b6f27af89d698c0cfdf6"
server_key_iv: "09384bcd04e43b22a5d4ec7048699012"
server_key_version: "encryptionkey@0"
I20260810 19:37:48.311798 22586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "f244abf270d143de8fdbad15cb4421ed"
format_stamp: "Formatted at 2026-08-10 19:37:48 on dist-test-slave-tm1g"
server_key: "e9c369ca2908b6f27af89d698c0cfdf6"
server_key_iv: "09384bcd04e43b22a5d4ec7048699012"
server_key_version: "encryptionkey@0"
I20260810 19:37:48.316941 22586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 19:37:48.320483 22602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:48.321907 22586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 19:37:48.322110 22586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "f244abf270d143de8fdbad15cb4421ed"
format_stamp: "Formatted at 2026-08-10 19:37:48 on dist-test-slave-tm1g"
server_key: "e9c369ca2908b6f27af89d698c0cfdf6"
server_key_iv: "09384bcd04e43b22a5d4ec7048699012"
server_key_version: "encryptionkey@0"
I20260810 19:37:48.322369 22586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:48.361364 22586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:48.365985 22586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:48.366315 22586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:48.385996 22586 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:41097
I20260810 19:37:48.386082 22657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:41097 every 8 connection(s)
I20260810 19:37:48.387383 22586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:48.391235 22658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:48.396390 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22586
I20260810 19:37:48.396579 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:48.397114 30630 external_mini_cluster.cc:1428] Setting key c3e943e003229cd850d2b743a626d7dc
I20260810 19:37:48.402734 22658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed: Bootstrap starting.
I20260810 19:37:48.405781 22658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:48.406915 22658 log.cc:824] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:48.410018 22658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed: No bootstrap required, opened a new log
I20260810 19:37:48.410424 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:48.399077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54351 (local address 127.29.233.190:41097)
0810 19:37:48.399921 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:48.399933 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:48.399975 (+ 42us) server_negotiation.cc:407] Connection header received
0810 19:37:48.401688 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:48.401710 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:48.402088 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:48.402519 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:48.403059 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:48.404382 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:48.405738 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:48.406540 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:48.406734 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:48.408032 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:48.408048 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:48.408074 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:48.408667 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:48.408739 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:48.408764 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:48.408852 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:48.409103 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 19:37:48.409361 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":401,"threads_started":1}
I20260810 19:37:48.414149 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:48.419957 22658 raft_consensus.cc:348] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f244abf270d143de8fdbad15cb4421ed" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41097 } }
I20260810 19:37:48.420352 22658 raft_consensus.cc:374] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:48.420491 22658 raft_consensus.cc:729] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f244abf270d143de8fdbad15cb4421ed, State: Initialized, Role: FOLLOWER
I20260810 19:37:48.421149 22658 consensus_queue.cc:260] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f244abf270d143de8fdbad15cb4421ed" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41097 } }
I20260810 19:37:48.421418 22658 raft_consensus.cc:388] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:48.421584 22658 raft_consensus.cc:482] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:48.421769 22658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:48.424167 22658 raft_consensus.cc:504] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f244abf270d143de8fdbad15cb4421ed" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41097 } }
I20260810 19:37:48.424705 22658 leader_election.cc:302] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f244abf270d143de8fdbad15cb4421ed; no voters:
I20260810 19:37:48.425350 22658 leader_election.cc:288] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:48.425530 22669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:48.426473 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:48.419640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32791 (local address 127.29.233.190:41097)
0810 19:37:48.419828 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:48.419834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:48.420402 (+ 568us) server_negotiation.cc:407] Connection header received
0810 19:37:48.420621 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:48.420625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:48.420685 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:48.420791 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:48.421343 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:48.422247 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:48.423606 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:48.424071 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:48.424244 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:48.425496 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:48.425503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:48.425508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:48.425533 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:48.425582 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:48.425605 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:48.425717 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:48.426003 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 19:37:48.426117 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:37:48.427863 22658 sys_catalog.cc:564] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:48.428659 22669 raft_consensus.cc:686] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [term 1 LEADER]: Becoming Leader. State: Replica: f244abf270d143de8fdbad15cb4421ed, State: Running, Role: LEADER
I20260810 19:37:48.429442 22669 consensus_queue.cc:237] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f244abf270d143de8fdbad15cb4421ed" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41097 } }
I20260810 19:37:48.432475 22671 sys_catalog.cc:455] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader f244abf270d143de8fdbad15cb4421ed. Latest consensus state: current_term: 1 leader_uuid: "f244abf270d143de8fdbad15cb4421ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f244abf270d143de8fdbad15cb4421ed" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41097 } } }
I20260810 19:37:48.432798 22671 sys_catalog.cc:458] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:48.433382 22670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f244abf270d143de8fdbad15cb4421ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f244abf270d143de8fdbad15cb4421ed" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 41097 } } }
I20260810 19:37:48.433686 22679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:48.433686 22670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:48.447613 22679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:48.458241 22679 catalog_manager.cc:1347] Generated new cluster ID: 64eeb7a33e94427c88ab2b193bf77269
I20260810 19:37:48.458400 22679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:48.469904 22679 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:48.471387 22679 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:48.483600 22679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f244abf270d143de8fdbad15cb4421ed: Generated new TSK 0
I20260810 19:37:48.484701 22679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:48.499260 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22586
W20260810 19:37:48.500594 22653 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54351
W20260810 19:37:48.501274 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22586: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:48.501430 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22586
2026-08-10T19:37:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2150 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:48 dist-test-slave-tm1g krb5kdc[22691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:48 dist-test-slave-tm1g krb5kdc[22691](info): set up 2 sockets
Aug 10 19:37:48 dist-test-slave-tm1g krb5kdc[22691](info): commencing operation
krb5kdc: starting...
W20260810 19:37:49.695263 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:49 dist-test-slave-tm1g krb5kdc[22691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:50.037963 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39575
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:39575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:50.391588 22705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:50.391871 22705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:50.391968 22705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:50.399065 22705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:50.399170 22705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:50.399217 22705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:50.399286 22705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:50.399323 22705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:50.409091 22705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:39575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:50.410369 22705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:50.411765 22705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:50.423797 22710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:50.423761 22713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:50.423928 22711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:50.424499 22705 server_base.cc:1034] running on GCE node
I20260810 19:37:50.425310 22705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:50.426815 22705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:50.428061 22705 hybrid_clock.cc:648] HybridClock initialized: now 1786390670428009 us; error 47 us; skew 500 ppm
I20260810 19:37:50.428417 22705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:50 dist-test-slave-tm1g krb5kdc[22691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390670, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:50.431710 22705 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:50.433168 22705 webserver.cc:460] Webserver started at http://127.29.233.190:35841/ using document root <none> and password file <none>
I20260810 19:37:50.433900 22705 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:50.434068 22705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:50.434449 22705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:50.437454 22705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "03818022db9547328ab82271f3a99839"
format_stamp: "Formatted at 2026-08-10 19:37:50 on dist-test-slave-tm1g"
server_key: "5f152d1d914eaa494b741efeed02ced9"
server_key_iv: "6640836127e6a3b8b6d4ae7355652eda"
server_key_version: "encryptionkey@0"
I20260810 19:37:50.438252 22705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "03818022db9547328ab82271f3a99839"
format_stamp: "Formatted at 2026-08-10 19:37:50 on dist-test-slave-tm1g"
server_key: "5f152d1d914eaa494b741efeed02ced9"
server_key_iv: "6640836127e6a3b8b6d4ae7355652eda"
server_key_version: "encryptionkey@0"
I20260810 19:37:50.443454 22705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 19:37:50.446970 22721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:50.448261 22705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:50.448455 22705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "03818022db9547328ab82271f3a99839"
format_stamp: "Formatted at 2026-08-10 19:37:50 on dist-test-slave-tm1g"
server_key: "5f152d1d914eaa494b741efeed02ced9"
server_key_iv: "6640836127e6a3b8b6d4ae7355652eda"
server_key_version: "encryptionkey@0"
I20260810 19:37:50.448659 22705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:50.454519 22705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:50.458855 22705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:50.459165 22705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:50.478091 22705 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:39575
I20260810 19:37:50.478135 22776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:39575 every 8 connection(s)
I20260810 19:37:50.479578 22705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:50.481209 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22705
I20260810 19:37:50.481405 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:50.481860 30630 external_mini_cluster.cc:1428] Setting key 753f0737bb648063615e34d4c728e4f3
I20260810 19:37:50.483917 22777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:50.489682 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:50.483857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46963 (local address 127.29.233.190:39575)
0810 19:37:50.484656 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:50.484668 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:50.484713 (+ 45us) server_negotiation.cc:407] Connection header received
0810 19:37:50.486393 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:50.486414 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:50.486746 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:50.487067 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:50.488161 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:50.488167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:50.488179 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:50.488193 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:50.488268 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:50.488288 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:50.488476 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:50.488573 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 19:37:50.488776 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":410,"threads_started":1}
I20260810 19:37:50.493340 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:50.496381 22777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839: Bootstrap starting.
I20260810 19:37:50.499784 22777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:50.500479 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:50.497867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51593 (local address 127.29.233.190:39575)
0810 19:37:50.498080 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:50.498088 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:50.498534 (+ 446us) server_negotiation.cc:407] Connection header received
0810 19:37:50.498714 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:50.498719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:50.498772 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:50.498874 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:50.499821 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:50.499827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:50.499831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 19:37:50.499848 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:50.499917 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 19:37:50.499944 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:50.500056 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:50.500163 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 19:37:50.500224 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 19:37:50.501036 22777 log.cc:824] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:50.503815 22777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839: No bootstrap required, opened a new log
I20260810 19:37:50.510967 22777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03818022db9547328ab82271f3a99839" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39575 } }
I20260810 19:37:50.511341 22777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 19:37:50.511463 22777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03818022db9547328ab82271f3a99839, State: Initialized, Role: FOLLOWER
I20260810 19:37:50.512060 22777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03818022db9547328ab82271f3a99839" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39575 } }
I20260810 19:37:50.512310 22777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 19:37:50.512465 22777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 19:37:50.512650 22777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:50.514912 22777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03818022db9547328ab82271f3a99839" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39575 } }
I20260810 19:37:50.515424 22777 leader_election.cc:302] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03818022db9547328ab82271f3a99839; no voters:
I20260810 19:37:50.516017 22777 leader_election.cc:288] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 19:37:50.516108 22787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:50.516911 22787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [term 1 LEADER]: Becoming Leader. State: Replica: 03818022db9547328ab82271f3a99839, State: Running, Role: LEADER
I20260810 19:37:50.517546 22787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03818022db9547328ab82271f3a99839" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39575 } }
I20260810 19:37:50.518218 22777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:50.520805 22788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03818022db9547328ab82271f3a99839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03818022db9547328ab82271f3a99839" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39575 } } }
I20260810 19:37:50.520931 22789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03818022db9547328ab82271f3a99839. Latest consensus state: current_term: 1 leader_uuid: "03818022db9547328ab82271f3a99839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03818022db9547328ab82271f3a99839" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 39575 } } }
I20260810 19:37:50.521240 22788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:50.521320 22789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:50.522327 22794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:50.528474 22794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:50.537662 22794 catalog_manager.cc:1347] Generated new cluster ID: ea3f47150ae241019599ebfb379aef1b
I20260810 19:37:50.537771 22794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:50.555044 22794 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:50.555552 22794 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:50.566179 22794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03818022db9547328ab82271f3a99839: Generated new TSK 0
I20260810 19:37:50.567209 22794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:50.579314 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22705
W20260810 19:37:50.580399 22772 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46963
W20260810 19:37:50.581110 30630 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22705: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 19:37:50.581277 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22705
2026-08-10T19:37:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2079 ms)
[----------] 18 tests from AuthTokenIssuingTest (38403 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:50 dist-test-slave-tm1g krb5kdc[22809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:50 dist-test-slave-tm1g krb5kdc[22809](info): set up 2 sockets
Aug 10 19:37:50 dist-test-slave-tm1g krb5kdc[22809](info): commencing operation
krb5kdc: starting...
W20260810 19:37:51.773090 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:51 dist-test-slave-tm1g krb5kdc[22809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:52.122342 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:46603
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46603,127.29.233.189:32959,127.29.233.188:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:52.471729 22824 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 19:37:52.472177 22824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:52.472325 22824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:52.472488 22824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:52.479820 22824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:52.479956 22824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:52.480072 22824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:52.480182 22824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:52.480279 22824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:52.490470 22824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46603,127.29.233.189:32959,127.29.233.188:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:46603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:52.491842 22824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:52.493227 22824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:52.505446 22830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:52.505555 22831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:52.505530 22833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:52.506415 22824 server_base.cc:1034] running on GCE node
I20260810 19:37:52.507220 22824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:52.508644 22824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:52.509908 22824 hybrid_clock.cc:648] HybridClock initialized: now 1786390672509834 us; error 58 us; skew 500 ppm
I20260810 19:37:52.510258 22824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:52 dist-test-slave-tm1g krb5kdc[22809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:52.517102 22824 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:52.518693 22824 webserver.cc:460] Webserver started at http://127.29.233.190:45811/ using document root <none> and password file <none>
I20260810 19:37:52.519469 22824 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:52.519609 22824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:52.519953 22824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:52.523126 22824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "2c66ed05d3f149ac8fa59ea1857c73b5"
format_stamp: "Formatted at 2026-08-10 19:37:52 on dist-test-slave-tm1g"
server_key: "6b4d422ea48ea95ec548a481bbafc002"
server_key_iv: "7ec9d66127e0a38fa37e330d3e490aab"
server_key_version: "encryptionkey@0"
I20260810 19:37:52.523976 22824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "2c66ed05d3f149ac8fa59ea1857c73b5"
format_stamp: "Formatted at 2026-08-10 19:37:52 on dist-test-slave-tm1g"
server_key: "6b4d422ea48ea95ec548a481bbafc002"
server_key_iv: "7ec9d66127e0a38fa37e330d3e490aab"
server_key_version: "encryptionkey@0"
I20260810 19:37:52.529398 22824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:37:52.532974 22841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:52.534524 22824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 19:37:52.534715 22824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "2c66ed05d3f149ac8fa59ea1857c73b5"
format_stamp: "Formatted at 2026-08-10 19:37:52 on dist-test-slave-tm1g"
server_key: "6b4d422ea48ea95ec548a481bbafc002"
server_key_iv: "7ec9d66127e0a38fa37e330d3e490aab"
server_key_version: "encryptionkey@0"
I20260810 19:37:52.534952 22824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:52.551820 22824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:52.556082 22824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:52.556376 22824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:52.576363 22824 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:46603
I20260810 19:37:52.576385 22896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:46603 every 8 connection(s)
I20260810 19:37:52.577734 22824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:52.581590 22897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:52.586521 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22824
I20260810 19:37:52.586763 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:52.587633 30630 external_mini_cluster.cc:1428] Setting key 416768048ea48374ef628eab9185ea28
I20260810 19:37:52.589577 22897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:37:52 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:52.613127 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.593575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46603 (local address 127.0.0.1:56654)
0810 19:37:52.594476 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.594530 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:52.596548 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:52.597368 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:52.597374 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:52.598091 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:52.598502 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:52.598520 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:52.600122 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:52.600130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:52.601202 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:52.601214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:52.602143 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:52.602151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:52.602373 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:52.603069 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:52.603090 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:52.605516 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:52.608688 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:52.608702 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:52.608714 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:52.609131 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:52.609563 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:52.609568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:52.609572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:52.609735 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:52.610583 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:52.610591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:52.610801 (+ 210us) client_negotiation.cc:769] Sending connection context
0810 19:37:52.611060 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 19:37:52.611725 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":214,"thread_start_us":392,"threads_started":1}
I20260810 19:37:52.613190 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.593576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56654 (local address 127.29.233.190:46603)
0810 19:37:52.595311 (+ 1735us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:52.595321 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:52.595367 (+ 46us) server_negotiation.cc:407] Connection header received
0810 19:37:52.596918 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:52.596923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:52.597138 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:52.597344 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:52.598726 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:52.599922 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:52.601356 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:52.601900 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:52.602127 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:52.605749 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:52.605757 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:52.605768 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:52.605795 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:52.608551 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:52.609284 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:52.609290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:52.609293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:52.609435 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:52.609859 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:52.609865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:52.609873 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:52.610427 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:52.610649 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:52.611182 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 19:37:52.611608 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":1500,"thread_start_us":1277,"threads_started":1}
I20260810 19:37:52.616228 22897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:52.617326 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.616922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57120)
0810 19:37:52.617097 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.617215 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 19:37:52.617704 22844 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
I20260810 19:37:52.619562 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.619077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57122)
0810 19:37:52.619273 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.619365 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260810 19:37:52.619858 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:32959: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.233.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab.
Entry for principal kudu/127.29.233.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab.
I20260810 19:37:52.665807 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:32959
--webserver_interface=127.29.233.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab
--principal=kudu/127.29.233.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46603,127.29.233.189:32959,127.29.233.188:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:37:52.669224 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } attempt: 1
I20260810 19:37:52.670555 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.670023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57138)
0810 19:37:52.670231 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.670408 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 19:37:52.671594 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.671216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57146)
0810 19:37:52.671371 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.671443 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 19:37:52.671789 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:32959: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
I20260810 19:37:52.740043 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } attempt: 2
I20260810 19:37:52.741433 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.740783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57162)
0810 19:37:52.741149 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.741257 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260810 19:37:52.745020 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.742480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57170)
0810 19:37:52.744695 (+ 2215us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.744811 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 19:37:52.745405 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:32959: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
I20260810 19:37:52.836704 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } attempt: 3
I20260810 19:37:52.838019 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.837506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57174)
0810 19:37:52.837770 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.837870 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260810 19:37:52.839200 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.838812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57176)
0810 19:37:52.838974 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.839056 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 19:37:52.839443 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:32959: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
I20260810 19:37:52.982733 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } attempt: 4
I20260810 19:37:52.984069 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.983512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57192)
0810 19:37:52.983831 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.983920 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260810 19:37:52.985087 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:52.984738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57208)
0810 19:37:52.984881 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:52.984973 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 19:37:52.985301 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:32959: Network error: Client connection negotiation failed: client connection to 127.29.233.189:32959: connect: Connection refused (error 111)
W20260810 19:37:53.025067 22905 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 19:37:53.025494 22905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:53.025599 22905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:53.025771 22905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:53.033325 22905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:53.033424 22905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:53.033530 22905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:53.033612 22905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:53.033727 22905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:53.044075 22905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46603,127.29.233.189:32959,127.29.233.188:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab
--principal=kudu/127.29.233.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:32959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.233.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:53.045526 22905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:53.046937 22905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:53.059213 22910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:53.059244 22911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:53.059296 22914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:53.060902 22905 server_base.cc:1034] running on GCE node
I20260810 19:37:53.061826 22905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:53.063347 22905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:53.064612 22905 hybrid_clock.cc:648] HybridClock initialized: now 1786390673064524 us; error 56 us; skew 500 ppm
I20260810 19:37:53.064976 22905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:53.068274 22905 init.cc:377] Logged in from keytab as kudu/127.29.233.189@KRBTEST.COM (short username kudu)
I20260810 19:37:53.069840 22905 webserver.cc:460] Webserver started at http://127.29.233.189:42917/ using document root <none> and password file <none>
I20260810 19:37:53.070673 22905 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:53.070822 22905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:53.071161 22905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:53.074154 22905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data/instance:
uuid: "d1451788947a4e1c9f633bd1c2218739"
format_stamp: "Formatted at 2026-08-10 19:37:53 on dist-test-slave-tm1g"
server_key: "61b01a249c35750df8c9578e09da1181"
server_key_iv: "392f332d88b0e9b423635fdeb8a7e3d3"
server_key_version: "encryptionkey@0"
I20260810 19:37:53.075075 22905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/wal/instance:
uuid: "d1451788947a4e1c9f633bd1c2218739"
format_stamp: "Formatted at 2026-08-10 19:37:53 on dist-test-slave-tm1g"
server_key: "61b01a249c35750df8c9578e09da1181"
server_key_iv: "392f332d88b0e9b423635fdeb8a7e3d3"
server_key_version: "encryptionkey@0"
I20260810 19:37:53.080260 22905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:37:53.083772 22923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:53.085103 22905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:53.085319 22905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/wal
uuid: "d1451788947a4e1c9f633bd1c2218739"
format_stamp: "Formatted at 2026-08-10 19:37:53 on dist-test-slave-tm1g"
server_key: "61b01a249c35750df8c9578e09da1181"
server_key_iv: "392f332d88b0e9b423635fdeb8a7e3d3"
server_key_version: "encryptionkey@0"
I20260810 19:37:53.085543 22905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:53.097277 22905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:53.101157 22905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:53.101439 22905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:53.121728 22905 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.189:32959
I20260810 19:37:53.121675 22976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.189:32959 every 8 connection(s)
I20260810 19:37:53.123176 22905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
I20260810 19:37:53.126947 22977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:53.130357 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22905
I20260810 19:37:53.130582 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-1/wal/instance
I20260810 19:37:53.131058 30630 external_mini_cluster.cc:1428] Setting key 4b9a300eb61f5f27d2e37da423f03bab
I20260810 19:37:53.133598 22977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:53.154421 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.136260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.29.233.190:46603)
0810 19:37:53.136862 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.136870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.137779 (+ 909us) server_negotiation.cc:407] Connection header received
0810 19:37:53.139862 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.139869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.139967 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.140094 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.141617 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.142467 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.143981 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.144395 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.144537 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.148302 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.148307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.148311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.148347 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.151278 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.152053 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.152060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.152064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.152189 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.152611 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.152616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.152620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.152907 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.153079 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.153778 (+ 699us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.154015 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":294,"threads_started":1}
I20260810 19:37:53.155387 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.136780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46603 (local address 127.0.0.1:56660)
0810 19:37:53.137588 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.137632 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.139542 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.140325 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.140334 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.140836 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.141389 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.141405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.142631 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.142635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.143811 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.143823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.144634 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.144641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.144871 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.145614 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.145637 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.148063 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.151432 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.151441 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.151484 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.151931 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.152307 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.152312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.152316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.152496 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.153049 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.153057 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.153418 (+ 361us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.153698 (+ 280us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.154203 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":339,"threads_started":1}
I20260810 19:37:53.157305 22977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.233.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.188@KRBTEST.COM" created.
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:53.176119 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.158012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57210)
0810 19:37:53.158282 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.158326 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.158433 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.159491 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.159499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.160021 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.160250 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.160265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.161757 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.161765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.163324 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.163342 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.164182 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.164195 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.164381 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.165091 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.165108 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.169001 (+ 3893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.173443 (+ 4442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.173447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.173450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.173841 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.174434 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.174438 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.174440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.174615 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.175397 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.175400 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.175541 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.175647 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.175852 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260810 19:37:53.176597 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.158294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.29.233.189:32959)
0810 19:37:53.159076 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.159081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.159109 (+ 28us) server_negotiation.cc:407] Connection header received
0810 19:37:53.159153 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.159158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.159314 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.159429 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.160436 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.161607 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.163479 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.164015 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.164111 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.169239 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.169256 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.169276 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.169321 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.173270 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.173997 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.174010 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.174016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.174165 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.174748 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.174755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.174761 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.175267 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.175496 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.175866 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.176167 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":421,"threads_started":1}
I20260810 19:37:53.179208 22977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:53.180708 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.180142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46382)
0810 19:37:53.180412 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.180553 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260810 19:37:53.181010 22927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.182694 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.182290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46396)
0810 19:37:53.182508 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.182591 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 19:37:53.183034 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab.
Entry for principal kudu/127.29.233.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab.
I20260810 19:37:53.210932 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:37125
--webserver_interface=127.29.233.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab
--principal=kudu/127.29.233.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:46603,127.29.233.189:32959,127.29.233.188:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:37:53.232401 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 1
I20260810 19:37:53.233824 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.233170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46400)
0810 19:37:53.233450 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.233596 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 19:37:53.235080 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.234565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46404)
0810 19:37:53.234780 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.234915 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 19:37:53.235303 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.284626 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } attempt: 5
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:53.300387 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.285611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57226 (local address 127.29.233.189:32959)
0810 19:37:53.285793 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.285798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.285817 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:37:53.286030 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.286034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.286113 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.286197 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.287318 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.288036 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.289833 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.290589 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.290689 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.294723 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.294728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.294731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.294763 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.297542 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.298436 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.298445 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.298452 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.298596 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.299049 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.299052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.299054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.299347 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.299463 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.299970 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.300150 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 19:37:53.300571 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.285461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57226)
0810 19:37:53.285711 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.285734 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.285888 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.286285 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.286290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.286883 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.287137 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.287151 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.288200 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.288208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.289658 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.289675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.290748 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.290754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.291008 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.291656 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.291680 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.294490 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.297725 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.297731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.297736 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.298280 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.298728 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.298733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.298737 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.298929 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.299489 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.299496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.299693 (+ 197us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.299870 (+ 177us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.300193 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 19:37:53.301937 22897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:53.303222 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.302675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46414)
0810 19:37:53.302913 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.303029 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 19:37:53.303532 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 2
I20260810 19:37:53.304472 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.304052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46426)
0810 19:37:53.304258 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.304341 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 19:37:53.304498 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.304011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46418)
0810 19:37:53.304228 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.304335 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260810 19:37:53.304888 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.305528 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.305164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46434)
0810 19:37:53.305334 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.305418 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 19:37:53.305783 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.356163 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 1
I20260810 19:37:53.357491 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.356891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46436)
0810 19:37:53.357152 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.357273 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 19:37:53.358714 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.358239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46452)
0810 19:37:53.358480 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.358577 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260810 19:37:53.358992 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.397006 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 3
I20260810 19:37:53.398398 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.397740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46460)
0810 19:37:53.398024 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.398144 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 19:37:53.399667 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.399162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46464)
0810 19:37:53.399420 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.399524 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260810 19:37:53.399931 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.427253 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 2
I20260810 19:37:53.428783 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.428071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46478)
0810 19:37:53.428388 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.428528 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260810 19:37:53.430253 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.429675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46482)
0810 19:37:53.429943 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.430074 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 19:37:53.430603 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.536895 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 3
I20260810 19:37:53.538407 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.537713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46490)
0810 19:37:53.537998 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.538125 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260810 19:37:53.539630 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.539141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46504)
0810 19:37:53.539360 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.539469 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260810 19:37:53.539877 22897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
I20260810 19:37:53.543128 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 4
I20260810 19:37:53.544344 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.543730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46514)
0810 19:37:53.544003 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.544120 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 19:37:53.545469 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.545080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46516)
0810 19:37:53.545240 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.545340 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 19:37:53.545691 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:37125: Network error: Client connection negotiation failed: client connection to 127.29.233.188:37125: connect: Connection refused (error 111)
W20260810 19:37:53.573808 22985 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 19:37:53.574153 22985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:53.574282 22985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:53.574404 22985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:53.581792 22985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:53.581872 22985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:53.581962 22985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:53.582055 22985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:53.582149 22985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:53.591996 22985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:46603,127.29.233.189:32959,127.29.233.188:37125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab
--principal=kudu/127.29.233.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.233.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:53.593210 22985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:53.594646 22985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:53.606377 22990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:53.606348 22991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:53.606546 22993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:53.607204 22985 server_base.cc:1034] running on GCE node
I20260810 19:37:53.607913 22985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:53.609213 22985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:53.610453 22985 hybrid_clock.cc:648] HybridClock initialized: now 1786390673610360 us; error 46 us; skew 500 ppm
I20260810 19:37:53.610741 22985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:53.613823 22985 init.cc:377] Logged in from keytab as kudu/127.29.233.188@KRBTEST.COM (short username kudu)
I20260810 19:37:53.615361 22985 webserver.cc:460] Webserver started at http://127.29.233.188:37339/ using document root <none> and password file <none>
I20260810 19:37:53.616076 22985 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:53.616214 22985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:53.616542 22985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:53.619526 22985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data/instance:
uuid: "8f27ff634aa74eecb2c55b5424d64c69"
format_stamp: "Formatted at 2026-08-10 19:37:53 on dist-test-slave-tm1g"
server_key: "04ec227c38ac034b757d4cc46a59ca47"
server_key_iv: "ae807bc7d13ac66db2d4a24d6ff3ac47"
server_key_version: "encryptionkey@0"
I20260810 19:37:53.620344 22985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/wal/instance:
uuid: "8f27ff634aa74eecb2c55b5424d64c69"
format_stamp: "Formatted at 2026-08-10 19:37:53 on dist-test-slave-tm1g"
server_key: "04ec227c38ac034b757d4cc46a59ca47"
server_key_iv: "ae807bc7d13ac66db2d4a24d6ff3ac47"
server_key_version: "encryptionkey@0"
I20260810 19:37:53.625424 22985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 19:37:53.628883 23000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:53.630160 22985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 19:37:53.630419 22985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/wal
uuid: "8f27ff634aa74eecb2c55b5424d64c69"
format_stamp: "Formatted at 2026-08-10 19:37:53 on dist-test-slave-tm1g"
server_key: "04ec227c38ac034b757d4cc46a59ca47"
server_key_iv: "ae807bc7d13ac66db2d4a24d6ff3ac47"
server_key_version: "encryptionkey@0"
I20260810 19:37:53.630648 22985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:53.664458 22985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:53.668605 22985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:53.668905 22985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:53.688179 22985 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.188:37125
I20260810 19:37:53.688275 23057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.188:37125 every 8 connection(s)
I20260810 19:37:53.689589 22985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
I20260810 19:37:53.693334 23058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:53.696120 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 22985
I20260810 19:37:53.696316 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786390305122638-30630-0/minicluster-data/master-2/wal/instance
I20260810 19:37:53.696863 30630 external_mini_cluster.cc:1428] Setting key 2ec60856128629615f5766ee4073e06d
I20260810 19:37:53.701567 23058 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:53.717131 22897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 4
I20260810 19:37:53.720659 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.703990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.29.233.190:46603)
0810 19:37:53.704523 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.704529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.705031 (+ 502us) server_negotiation.cc:407] Connection header received
0810 19:37:53.705211 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.705215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.705300 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.705404 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.706789 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.707895 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.709666 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.710074 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.710216 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.713992 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.713997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.714000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.714039 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.717293 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.718210 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.718218 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.718221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.718401 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.718896 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.718902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.718905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.719299 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.719517 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.720056 (+ 539us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.720271 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":245,"threads_started":1}
I20260810 19:37:53.722041 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.698927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.29.233.188:37125)
0810 19:37:53.699839 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.699852 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.699892 (+ 40us) server_negotiation.cc:407] Connection header received
0810 19:37:53.701501 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.701520 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.701894 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.702276 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.703976 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.706099 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.707829 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.708868 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.709042 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.713579 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.713597 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.713614 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.713652 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.716414 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.717400 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.717411 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.717419 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.717636 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.718129 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.718137 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.718141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.719085 (+ 944us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.720650 (+ 1565us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.720849 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.721100 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":388,"threads_started":1}
I20260810 19:37:53.722088 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.704038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:46603 (local address 127.0.0.1:56668)
0810 19:37:53.704848 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.704904 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.705090 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.705508 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.705516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.706357 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.706624 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.706637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.708049 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.708054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.709513 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.709523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.710350 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.710359 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.710532 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.711157 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.711178 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.713785 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.717442 (+ 3657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.717463 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.717472 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.718070 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.718601 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.718610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.718613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.718765 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.719438 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.719448 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.719674 (+ 226us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.719941 (+ 267us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.720480 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":413,"threads_started":1}
I20260810 19:37:53.723994 23058 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:53.734942 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.717759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46534)
0810 19:37:53.718029 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.718052 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.718159 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.719120 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.719128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.719693 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.719928 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.719941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.721636 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.721642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.723171 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.723186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.724193 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.724199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.724431 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.725080 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.725105 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.728005 (+ 2900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.732042 (+ 4037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.732047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.732052 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.732492 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.732893 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.732901 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.732904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.733074 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.733816 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.733822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.734014 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.734190 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.734504 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260810 19:37:53.735006 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.717967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46534 (local address 127.29.233.188:37125)
0810 19:37:53.718760 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.718764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.718782 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:37:53.718817 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.718849 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.718923 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.719037 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.720146 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.721487 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.723332 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.724030 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.724180 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.728252 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.728259 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.728264 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.728298 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.731897 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.732607 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.732614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.732617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.732785 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.733204 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.733212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.733215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.733563 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.733732 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.734345 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.734582 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":505,"threads_started":1}
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:53.740197 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.724646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:32959 (local address 127.0.0.1:57238)
0810 19:37:53.724870 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.724887 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.724983 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.725442 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.725449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.725865 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.726029 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.726040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.727343 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.727348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.728393 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.728406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.729368 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.729376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.729615 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.730180 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.730224 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.733268 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.737417 (+ 4149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.737422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.737425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.737933 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.738422 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.738427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.738431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.738610 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.739245 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.739249 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.739425 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.739607 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.739871 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 19:37:53.740464 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.724893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.29.233.189:32959)
0810 19:37:53.725103 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.725109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.725127 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:37:53.725186 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.725190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.725261 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.725388 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.726172 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.727118 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.728565 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.729213 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.729338 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.733458 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.733464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.733467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.733499 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.737255 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.738063 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.738072 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.738078 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.738264 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.738742 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.738748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.738752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.739084 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.739254 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.739725 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.739993 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 19:37:53.741478 23058 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:53.742827 22897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5: Bootstrap starting.
I20260810 19:37:53.747575 22897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:53.747792 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.725843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.29.233.190:46603)
0810 19:37:53.726025 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.726029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.726046 (+ 17us) server_negotiation.cc:407] Connection header received
0810 19:37:53.727348 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.727353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.727450 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.729098 (+ 1648us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.729230 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.730073 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.731380 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.731898 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.732667 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.737400 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.737410 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.737414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.737456 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.744370 (+ 6914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.745282 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.745338 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.745343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.745510 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.745966 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.745971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.745975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.746501 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.746796 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.747367 (+ 571us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.747544 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:37:53.749125 22897 log.cc:824] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:53.753181 22897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5: No bootstrap required, opened a new log
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:53.762354 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.742634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46542 (local address 127.29.233.188:37125)
0810 19:37:53.749782 (+ 7148us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.749789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.749811 (+ 22us) server_negotiation.cc:407] Connection header received
0810 19:37:53.749880 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.749884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.749980 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.750085 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.750874 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.751573 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.752726 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.753156 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.753318 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.756322 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.756327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.756330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.756356 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.759081 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.760241 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.760245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.760247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.760363 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.760731 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.760736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.760744 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.760983 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.761154 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.761550 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.761966 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":7028}
I20260810 19:37:53.762391 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.742218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46542)
0810 19:37:53.742794 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.742817 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.742917 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.750106 (+ 7189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.750110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.750591 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.750735 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.750744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.751698 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.751706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.752581 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.752591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.753337 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.753343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.753525 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.754121 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.754138 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.756138 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.759210 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.759218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.759222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.759592 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.760478 (+ 886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.760484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.760488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.760620 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.761101 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.761110 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.761256 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.761412 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.761748 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473}
I20260810 19:37:53.764626 22897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.765055 22897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c66ed05d3f149ac8fa59ea1857c73b5, State: Initialized, Role: FOLLOWER
I20260810 19:37:53.765574 22897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.766642 23066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.767035 23066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:53.767918 22897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:53.768181 23058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: Bootstrap starting.
I20260810 19:37:53.768658 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.749798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57240 (local address 127.29.233.189:32959)
0810 19:37:53.749996 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.750003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.750023 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:37:53.750079 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.750083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.750150 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.750248 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.752071 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.753150 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.754952 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.755642 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.755749 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.761434 (+ 5685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.761439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.761444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.761484 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.765177 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.766124 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.766130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.766133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.766290 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.766787 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.766795 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.766801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.767157 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.767419 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.768038 (+ 619us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.768287 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 19:37:53.772380 23058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:53.773747 23058 log.cc:824] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:53.774390 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:53.778960 22870 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56676
I20260810 19:37:53.779377 22870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 19:37:53.779657 22870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:53.782469 22870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.784112 22870 leader_election.cc:288] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [CANDIDATE]: Term 1 election: Requested vote from peers d1451788947a4e1c9f633bd1c2218739 (127.29.233.189:32959), 8f27ff634aa74eecb2c55b5424d64c69 (127.29.233.188:37125)
I20260810 19:37:53.787365 23058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: No bootstrap required, opened a new log
W20260810 19:37:53.790882 23078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 19:37:53.791013 23078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 19:37:53.779042 22824 master_runner.cc:186] Error getting master registration for 127.29.233.189:32959: OK, instance_id { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" instance_seqno: 1786390673116085 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 19:37:53.791620 22824 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 19:37:53.802127 23058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.802701 23058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f27ff634aa74eecb2c55b5424d64c69, State: Initialized, Role: FOLLOWER
I20260810 19:37:53.803345 23058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.805091 23085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.806178 23085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:53.806564 23032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8f27ff634aa74eecb2c55b5424d64c69"
I20260810 19:37:53.807192 23032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [term 0 FOLLOWER]: Advancing to term 1
W20260810 19:37:53.809325 22847 tablet.cc:2369] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:53.811489 23032 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c66ed05d3f149ac8fa59ea1857c73b5 in term 1.
I20260810 19:37:53.812332 22845 leader_election.cc:302] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c66ed05d3f149ac8fa59ea1857c73b5, 8f27ff634aa74eecb2c55b5424d64c69; no voters:
I20260810 19:37:53.806536 23058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:53.812887 23066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:53.821171 23066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [term 1 LEADER]: Becoming Leader. State: Replica: 2c66ed05d3f149ac8fa59ea1857c73b5, State: Running, Role: LEADER
I20260810 19:37:53.822350 23066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.842360 23087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c66ed05d3f149ac8fa59ea1857c73b5. Latest consensus state: current_term: 1 leader_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.842835 23087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:53.844892 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } attempt: 5
I20260810 19:37:53.848975 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.822269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.29.233.189:32959)
0810 19:37:53.822515 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.822524 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.826515 (+ 3991us) server_negotiation.cc:407] Connection header received
0810 19:37:53.827357 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.827366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.827477 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.827599 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.830666 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.832155 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.834909 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.835839 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.835963 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.840311 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.840320 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.840328 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.840396 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.845567 (+ 5171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.846450 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.846457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.846460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.846623 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.847132 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.847138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.847142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.847536 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.847779 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.848338 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.848573 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Aug 10 19:37:53 dist-test-slave-tm1g krb5kdc[22809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
I20260810 19:37:53.858326 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.812321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.29.233.190:46603)
0810 19:37:53.812526 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.812532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.821933 (+ 9401us) server_negotiation.cc:407] Connection header received
0810 19:37:53.822525 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.822531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.822623 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.822704 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.828548 (+ 5844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.829913 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.833722 (+ 3809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.834598 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.834737 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.849659 (+ 14922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.849668 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.849675 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.849727 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.853392 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.854934 (+ 1542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.854942 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.854947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.855184 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.856028 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.856035 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.856039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.856404 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.856682 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.857638 (+ 956us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.857881 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 19:37:53.865393 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.845488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:37125 (local address 127.0.0.1:46568)
0810 19:37:53.845860 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:53.845880 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:53.845974 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:53.846822 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:53.846830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:53.847342 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:53.847548 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.847559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.848889 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.848894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.850283 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:53.850326 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.851264 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.851269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:53.851524 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.852282 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:53.852313 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:53.855900 (+ 3587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:53.861597 (+ 5697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.861606 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.861613 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.862336 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.862850 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:53.862858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:53.862864 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:53.863080 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:53.864024 (+ 944us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:53.864032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:53.864231 (+ 199us) client_negotiation.cc:769] Sending connection context
0810 19:37:53.864405 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 19:37:53.864788 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1}
I20260810 19:37:53.866667 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.846177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.29.233.188:37125)
0810 19:37:53.846380 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.846389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.846408 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:37:53.846472 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.846477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.846551 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.846643 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.847703 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.848731 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.850488 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.851093 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.851257 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.856212 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.856222 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.856229 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.856287 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.861362 (+ 5075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.862491 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.862498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.862504 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.862683 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.863238 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.863246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.863252 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.863760 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.863934 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.865743 (+ 1809us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.866145 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:37:53.867409 23096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:53.869740 23096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:53.873503 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:53.820583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.29.233.188:37125)
0810 19:37:53.820797 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:53.820804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:53.844215 (+ 23411us) server_negotiation.cc:407] Connection header received
0810 19:37:53.845224 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:53.845233 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:53.845384 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:53.845479 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:53.850871 (+ 5392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.852041 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.855947 (+ 3906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:53.856720 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:53.856829 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:53.862988 (+ 6159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:53.862996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:53.863000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:53.863038 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:53.868134 (+ 5096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.869627 (+ 1493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.869632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.869636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.869928 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:53.870791 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:53.870798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:53.870803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:53.871169 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:53.871499 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:53.872532 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0810 19:37:53.872920 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 19:37:53.874004 22977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739: Bootstrap starting.
I20260810 19:37:53.879225 22977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:53.880996 22977 log.cc:824] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:53.891343 22977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739: No bootstrap required, opened a new log
I20260810 19:37:53.894783 23032 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [term 1 FOLLOWER]: Refusing update from remote peer 2c66ed05d3f149ac8fa59ea1857c73b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:53.896389 23087 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:53.903939 22977 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.904498 22977 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1451788947a4e1c9f633bd1c2218739, State: Initialized, Role: FOLLOWER
I20260810 19:37:53.905074 22977 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } }
I20260810 19:37:53.918743 23085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c66ed05d3f149ac8fa59ea1857c73b5. Latest consensus state: current_term: 1 leader_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.919071 23085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 19:37:53.921002 23008 tablet.cc:2369] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:53.920791 23107 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.921275 23107 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:53.923830 23101 mvcc.cc:205] Tried to move back new op lower bound from 7317056200252968960 to 7317056199987855360. Current Snapshot: MvccSnapshot[applied={T|T < 7317056200252968960}]
I20260810 19:37:53.924665 22977 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:53.924715 22952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d1451788947a4e1c9f633bd1c2218739"
I20260810 19:37:53.925093 22951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:53.929931 22951 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [term 1 FOLLOWER]: Refusing update from remote peer 2c66ed05d3f149ac8fa59ea1857c73b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:53.932595 23066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.932943 23066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:53.935539 23096 catalog_manager.cc:1347] Generated new cluster ID: abf724b55e454e9bb6c374eb4c0d74d1
I20260810 19:37:53.935618 23096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:53.936995 22952 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c66ed05d3f149ac8fa59ea1857c73b5 in term 1.
I20260810 19:37:53.939699 23066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:53.941116 23085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.941359 23085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:53.935396 22985 master_runner.cc:186] Error getting master registration for 127.29.233.189:32959: OK, instance_id { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" instance_seqno: 1786390673116085 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 19:37:53.947118 22985 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 19:37:53.950465 23108 catalog_manager.cc:1259] Loaded cluster ID: abf724b55e454e9bb6c374eb4c0d74d1
I20260810 19:37:53.950624 23108 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: loading cluster ID for follower catalog manager: success
W20260810 19:37:53.953047 23108 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260810 19:37:53.957350 23108 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260810 19:37:53.981525 23096 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:53.983825 23096 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:53.997530 23107 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c66ed05d3f149ac8fa59ea1857c73b5. Latest consensus state: current_term: 1 leader_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.997864 23107 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:53.998401 23119 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:53.998812 23119 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:53.999852 23066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c66ed05d3f149ac8fa59ea1857c73b5" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 46603 } } peers { permanent_uuid: "d1451788947a4e1c9f633bd1c2218739" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 32959 } } peers { permanent_uuid: "8f27ff634aa74eecb2c55b5424d64c69" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 37125 } } }
I20260810 19:37:54.000120 23066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5 [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:54.017560 23127 catalog_manager.cc:1259] Loaded cluster ID: abf724b55e454e9bb6c374eb4c0d74d1
I20260810 19:37:54.017738 23127 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739: loading cluster ID for follower catalog manager: success
I20260810 19:37:54.021548 23127 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739: acquiring CA information for follower catalog manager: success
W20260810 19:37:54.023002 23127 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260810 19:37:54.025187 23096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c66ed05d3f149ac8fa59ea1857c73b5: Generated new TSK 0
I20260810 19:37:54.026213 23096 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 19:37:54.157704 30630 mini_kdc.cc:348] Time spent kdestroy: real 0.106s user 0.000s sys 0.031s
W20260810 19:37:54.209198 23062 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.197245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56700 (local address 127.29.233.190:46603)
0810 19:37:54.197472 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.197479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.206371 (+ 8892us) server_negotiation.cc:407] Connection header received
0810 19:37:54.207562 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.207570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.207662 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.207740 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.208915 (+ 1175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56700: BlockingRecv error: recv got EOF from 127.0.0.1:56700 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 19:37:54.213960 23064 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.200955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.29.233.188:37125)
0810 19:37:54.201144 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.201151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.210624 (+ 9473us) server_negotiation.cc:407] Connection header received
0810 19:37:54.212466 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.212473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.212561 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.212635 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.213675 (+ 1040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46578: BlockingRecv error: recv got EOF from 127.0.0.1:46578 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 19:37:54.216041 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.211498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.29.233.189:32959)
0810 19:37:54.211669 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.211675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.212181 (+ 506us) server_negotiation.cc:407] Connection header received
0810 19:37:54.214532 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.214538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.214635 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.214708 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.215822 (+ 1114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57258: BlockingRecv error: recv got EOF from 127.0.0.1:57258 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 19:37:54.257277 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.246974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57266 (local address 127.29.233.189:32959)
0810 19:37:54.247190 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.247196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.255051 (+ 7855us) server_negotiation.cc:407] Connection header received
0810 19:37:54.255642 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.255648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.255777 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.255862 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.256968 (+ 1106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57266: BlockingRecv error: recv got EOF from 127.0.0.1:57266 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 19:37:54.300251 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.291703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57278 (local address 127.29.233.189:32959)
0810 19:37:54.291895 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.291901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.298079 (+ 6178us) server_negotiation.cc:407] Connection header received
0810 19:37:54.298651 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.298658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.298757 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.298851 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.299990 (+ 1139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57278: BlockingRecv error: recv got EOF from 127.0.0.1:57278 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 19:37:54.347133 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.337059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.29.233.189:32959)
0810 19:37:54.337274 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.337281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.344880 (+ 7599us) server_negotiation.cc:407] Connection header received
0810 19:37:54.345470 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.345480 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.345595 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.345680 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.346856 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57280: BlockingRecv error: recv got EOF from 127.0.0.1:57280 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 19:37:54.402946 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.393109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.29.233.189:32959)
0810 19:37:54.393354 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.393361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.400882 (+ 7521us) server_negotiation.cc:407] Connection header received
0810 19:37:54.401406 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.401412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.401537 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.401624 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.402688 (+ 1064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57282: BlockingRecv error: recv got EOF from 127.0.0.1:57282 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260810 19:37:54.454193 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.444247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.29.233.189:32959)
0810 19:37:54.444469 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.444476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.452011 (+ 7535us) server_negotiation.cc:407] Connection header received
0810 19:37:54.452602 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.452609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.452714 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.452803 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.453920 (+ 1117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57298: BlockingRecv error: recv got EOF from 127.0.0.1:57298 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 19:37:54.486912 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.484337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.29.233.189:32959)
0810 19:37:54.484531 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.484537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.485162 (+ 625us) server_negotiation.cc:407] Connection header received
0810 19:37:54.485402 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.485411 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.485518 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.485659 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.486632 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57308: BlockingRecv error: recv got EOF from 127.0.0.1:57308 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 19:37:54.529862 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.526990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57322 (local address 127.29.233.189:32959)
0810 19:37:54.527203 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.527210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.527757 (+ 547us) server_negotiation.cc:407] Connection header received
0810 19:37:54.528099 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.528106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.528220 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.528360 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.529570 (+ 1210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57322: BlockingRecv error: recv got EOF from 127.0.0.1:57322 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 19:37:54.599996 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.598034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57328 (local address 127.29.233.189:32959)
0810 19:37:54.598256 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.598263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.598677 (+ 414us) server_negotiation.cc:407] Connection header received
0810 19:37:54.598882 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.598888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.598979 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.599087 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.599776 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57328: BlockingRecv error: recv got EOF from 127.0.0.1:57328 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 19:37:54.733464 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.731371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.29.233.189:32959)
0810 19:37:54.731639 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.731647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.731938 (+ 291us) server_negotiation.cc:407] Connection header received
0810 19:37:54.732192 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.732198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.732278 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.732395 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.733203 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57338: BlockingRecv error: recv got EOF from 127.0.0.1:57338 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 19:37:54.960943 23108 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: acquiring CA information for follower catalog manager: success
I20260810 19:37:54.962863 23108 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8f27ff634aa74eecb2c55b5424d64c69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 19:37:54.995371 22982 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:54.993264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.29.233.189:32959)
0810 19:37:54.993535 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:54.993542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:54.993752 (+ 210us) server_negotiation.cc:407] Connection header received
0810 19:37:54.994023 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:54.994027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:54.994102 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:54.994225 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:54.995122 (+ 897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57350: BlockingRecv error: recv got EOF from 127.0.0.1:57350 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 19:37:55.026492 23127 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d1451788947a4e1c9f633bd1c2218739: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 19:37:55.518456 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:55.511995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.29.233.189:32959)
0810 19:37:55.512560 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:55.512567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:55.512689 (+ 122us) server_negotiation.cc:407] Connection header received
0810 19:37:55.512965 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:55.512969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:55.513186 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:55.513321 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:55.513765 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.514680 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.516070 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.516508 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.516650 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:55.517168 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:55.517734 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:55.517903 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:55.518000 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 19:37:55.518089 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":272,"threads_started":1}
I20260810 19:37:55.530546 23205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:55.524586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.29.233.188:37125)
0810 19:37:55.525212 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:55.525218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:55.525237 (+ 19us) server_negotiation.cc:407] Connection header received
0810 19:37:55.525522 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:55.525527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:55.525679 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:55.525855 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:55.526262 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.527037 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.528314 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.528723 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.528857 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:55.529347 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:55.529909 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:55.530077 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:55.530134 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 19:37:55.530216 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":313,"threads_started":1}
I20260810 19:37:55.545065 23205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:55.537341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.29.233.188:37125)
0810 19:37:55.537533 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:55.537541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:55.538628 (+ 1087us) server_negotiation.cc:407] Connection header received
0810 19:37:55.538899 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:55.538905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:55.539005 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:55.539070 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:55.539859 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.540876 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.543285 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.543737 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.543885 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:55.544322 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:55.544424 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:55.544515 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:55.544674 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 19:37:55.544755 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 19:37:55.546118 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:55.536989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56716 (local address 127.29.233.190:46603)
0810 19:37:55.538755 (+ 1766us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:55.538762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:55.538782 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:37:55.538970 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:55.538976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:55.539206 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:55.539462 (+ 256us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:55.540127 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.541033 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.543294 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.543702 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.543798 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:55.544466 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:55.544950 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:55.545062 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:55.545575 (+ 513us) server_negotiation.cc:299] Negotiation successful
0810 19:37:55.545672 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1639,"thread_start_us":1530,"threads_started":1}
I20260810 19:37:55.546156 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:55.537616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.29.233.189:32959)
0810 19:37:55.537798 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:55.537804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:55.539219 (+ 1415us) server_negotiation.cc:407] Connection header received
0810 19:37:55.539423 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:55.539429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:55.539521 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:55.539614 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:55.540663 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.541536 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.543986 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:55.544590 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:55.544704 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:55.545321 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:55.545451 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:55.545518 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:55.545810 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 19:37:55.545898 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 19:37:55.557396 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22824
I20260810 19:37:55.558903 22892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:55.671404 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22824
I20260810 19:37:55.699775 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22905
I20260810 19:37:55.701057 22972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:55.812546 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22905
I20260810 19:37:55.837116 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 22985
I20260810 19:37:55.838282 23053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:37:55.947751 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 22985
2026-08-10T19:37:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5371 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 19:37:56 dist-test-slave-tm1g krb5kdc[23219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 19:37:56 dist-test-slave-tm1g krb5kdc[23219](info): set up 2 sockets
Aug 10 19:37:56 dist-test-slave-tm1g krb5kdc[23219](info): commencing operation
krb5kdc: starting...
W20260810 19:37:57.159664 30630 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/test-user.keytab.
Aug 10 19:37:57 dist-test-slave-tm1g krb5kdc[23219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T19:37:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T19:37:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.233.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
Entry for principal kudu/127.29.233.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab.
I20260810 19:37:57.513814 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35243
--webserver_interface=127.29.233.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:35243,127.29.233.189:36395,127.29.233.188:35821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 19:37:57.872802 23232 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 19:37:57.873148 23232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:57.873275 23232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:57.873394 23232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:57.880697 23232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:57.880791 23232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:57.880849 23232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:57.880893 23232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:57.880956 23232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:57.890914 23232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:35243,127.29.233.189:36395,127.29.233.188:35821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.190.keytab
--principal=kudu/127.29.233.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.190:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.233.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:57.892199 23232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:57.893510 23232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:57.905809 23237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:57.905886 23240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:57.905911 23238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:57.907334 23232 server_base.cc:1034] running on GCE node
I20260810 19:37:57.908070 23232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:57.909400 23232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:57.910621 23232 hybrid_clock.cc:648] HybridClock initialized: now 1786390677910560 us; error 34 us; skew 500 ppm
I20260810 19:37:57.910913 23232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:57 dist-test-slave-tm1g krb5kdc[23219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:57.919691 23232 init.cc:377] Logged in from keytab as kudu/127.29.233.190@KRBTEST.COM (short username kudu)
I20260810 19:37:57.921358 23232 webserver.cc:460] Webserver started at http://127.29.233.190:46263/ using document root <none> and password file <none>
I20260810 19:37:57.922127 23232 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:57.922240 23232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:57.922621 23232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:57.925719 23232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data/instance:
uuid: "8c34ee3175d8481c9c91f0b798ece62d"
format_stamp: "Formatted at 2026-08-10 19:37:57 on dist-test-slave-tm1g"
server_key: "47438cccae6ac4cd8ec7a9a88edbbd83"
server_key_iv: "917674111cbe243902402f9eef287960"
server_key_version: "encryptionkey@0"
I20260810 19:37:57.926618 23232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/wal/instance:
uuid: "8c34ee3175d8481c9c91f0b798ece62d"
format_stamp: "Formatted at 2026-08-10 19:37:57 on dist-test-slave-tm1g"
server_key: "47438cccae6ac4cd8ec7a9a88edbbd83"
server_key_iv: "917674111cbe243902402f9eef287960"
server_key_version: "encryptionkey@0"
I20260810 19:37:57.931982 23232 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 19:37:57.935595 23247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:57.936954 23232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 19:37:57.937155 23232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/wal
uuid: "8c34ee3175d8481c9c91f0b798ece62d"
format_stamp: "Formatted at 2026-08-10 19:37:57 on dist-test-slave-tm1g"
server_key: "47438cccae6ac4cd8ec7a9a88edbbd83"
server_key_iv: "917674111cbe243902402f9eef287960"
server_key_version: "encryptionkey@0"
I20260810 19:37:57.937373 23232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:57.952508 23232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:57.957054 23232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:57.957401 23232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:57.977419 23232 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.190:35243
I20260810 19:37:57.977469 23306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.190:35243 every 8 connection(s)
I20260810 19:37:57.978901 23232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/data/info.pb
I20260810 19:37:57.982736 23307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:57.988529 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 23232
I20260810 19:37:57.988761 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-0/wal/instance
I20260810 19:37:57.989352 30630 external_mini_cluster.cc:1428] Setting key 6d69a6e68440eee7a4ed8382a4f197a9
I20260810 19:37:57.989982 23307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:37:58 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:58.027143 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:57.993410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50618 (local address 127.29.233.190:35243)
0810 19:37:57.998963 (+ 5553us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:57.998975 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:58.000895 (+ 1920us) server_negotiation.cc:407] Connection header received
0810 19:37:58.003560 (+ 2665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:58.003571 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:58.003953 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:58.004182 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:58.006218 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.008007 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.010397 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.011373 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.011575 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.016532 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:58.016545 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:58.016565 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:58.016611 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:58.021065 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.022152 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.022163 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.022169 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.022390 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.022874 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.022882 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.022888 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.023683 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:58.023908 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:58.024787 (+ 879us) server_negotiation.cc:299] Negotiation successful
0810 19:37:58.025224 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":5337,"spinlock_wait_cycles":69888,"thread_start_us":5044,"threads_started":1}
I20260810 19:37:58.027191 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:57.993411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:35243 (local address 127.0.0.1:50618)
0810 19:37:57.999901 (+ 6490us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:57.999978 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:58.002960 (+ 2982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:58.004311 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:58.004322 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:58.005413 (+ 1091us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:58.005951 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.005981 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.008187 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.008194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.010184 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.010205 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.011646 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.011659 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.011937 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.012765 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:58.012805 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:58.016257 (+ 3452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:58.021235 (+ 4978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.021250 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.021268 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.021969 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.022508 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.022515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.022521 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.022717 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.023910 (+ 1193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:58.023920 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:58.024226 (+ 306us) client_negotiation.cc:769] Sending connection context
0810 19:37:58.024573 (+ 347us) client_negotiation.cc:240] Negotiation successful
0810 19:37:58.025462 (+ 889us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5828,"mutex_wait_us":198,"thread_start_us":5626,"threads_started":1}
I20260810 19:37:58.031018 23307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:58.032438 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.031998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56058)
0810 19:37:58.032182 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.032320 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.29.233.189@KRBTEST.COM; defaulting to no policy
W20260810 19:37:58.032792 23249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
I20260810 19:37:58.034706 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.034195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56070)
0810 19:37:58.034445 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.034552 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 19:37:58.034948 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:36395: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab.
Entry for principal kudu/127.29.233.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab.
I20260810 19:37:58.078195 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:36395
--webserver_interface=127.29.233.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab
--principal=kudu/127.29.233.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:35243,127.29.233.189:36395,127.29.233.188:35821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:37:58.084352 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } attempt: 1
I20260810 19:37:58.085743 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.085088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56080)
0810 19:37:58.085357 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.085544 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 19:37:58.086968 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.086486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56088)
0810 19:37:58.086665 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.086765 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 19:37:58.087199 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:36395: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
I20260810 19:37:58.155503 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } attempt: 2
I20260810 19:37:58.156798 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.156273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56092)
0810 19:37:58.156547 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.156650 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 19:37:58.157895 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.157522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56096)
0810 19:37:58.157689 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.157758 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 19:37:58.158087 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:36395: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
I20260810 19:37:58.249348 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } attempt: 3
I20260810 19:37:58.250720 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.250162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56098)
0810 19:37:58.250467 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.250566 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 19:37:58.251957 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.251552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:56106)
0810 19:37:58.251715 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.251790 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 19:37:58.252144 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:36395: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
I20260810 19:37:58.395427 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } attempt: 4
I20260810 19:37:58.396730 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.396147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:52422)
0810 19:37:58.396500 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.396589 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260810 19:37:58.397809 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.397459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:52436)
0810 19:37:58.397611 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.397678 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 19:37:58.398005 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.189:36395: Network error: Client connection negotiation failed: client connection to 127.29.233.189:36395: connect: Connection refused (error 111)
W20260810 19:37:58.431936 23314 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 19:37:58.432370 23314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:58.432505 23314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:58.432643 23314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:58.440166 23314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:58.440308 23314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:58.440428 23314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:58.440516 23314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:58.440608 23314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:58.451378 23314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:35243,127.29.233.189:36395,127.29.233.188:35821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.189.keytab
--principal=kudu/127.29.233.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.189:36395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.233.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:58.452859 23314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:58.454260 23314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:58.466701 23319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:58.466810 23320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:58.467300 23322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:58.468109 23314 server_base.cc:1034] running on GCE node
I20260810 19:37:58.468928 23314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:58.470556 23314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:58.471907 23314 hybrid_clock.cc:648] HybridClock initialized: now 1786390678471815 us; error 51 us; skew 500 ppm
I20260810 19:37:58.472347 23314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:58 dist-test-slave-tm1g krb5kdc[23219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390678, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:58.475745 23314 init.cc:377] Logged in from keytab as kudu/127.29.233.189@KRBTEST.COM (short username kudu)
I20260810 19:37:58.477285 23314 webserver.cc:460] Webserver started at http://127.29.233.189:42321/ using document root <none> and password file <none>
I20260810 19:37:58.478032 23314 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:58.478142 23314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:58.478535 23314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:58.481624 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data/instance:
uuid: "bc6270d168de4c748af1793b6b2fef63"
format_stamp: "Formatted at 2026-08-10 19:37:58 on dist-test-slave-tm1g"
server_key: "c82bcd352019b96a431b8ac5ce8a9631"
server_key_iv: "c8541c3625a8d572fd74f856dac869d2"
server_key_version: "encryptionkey@0"
I20260810 19:37:58.482507 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/wal/instance:
uuid: "bc6270d168de4c748af1793b6b2fef63"
format_stamp: "Formatted at 2026-08-10 19:37:58 on dist-test-slave-tm1g"
server_key: "c82bcd352019b96a431b8ac5ce8a9631"
server_key_iv: "c8541c3625a8d572fd74f856dac869d2"
server_key_version: "encryptionkey@0"
I20260810 19:37:58.487829 23314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 19:37:58.491355 23329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:58.492759 23314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 19:37:58.492944 23314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/wal
uuid: "bc6270d168de4c748af1793b6b2fef63"
format_stamp: "Formatted at 2026-08-10 19:37:58 on dist-test-slave-tm1g"
server_key: "c82bcd352019b96a431b8ac5ce8a9631"
server_key_iv: "c8541c3625a8d572fd74f856dac869d2"
server_key_version: "encryptionkey@0"
I20260810 19:37:58.493160 23314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:58.504491 23314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:58.508821 23314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:58.509097 23314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:58.528802 23314 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.189:36395
I20260810 19:37:58.528826 23381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.189:36395 every 8 connection(s)
I20260810 19:37:58.530225 23314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/data/info.pb
I20260810 19:37:58.533557 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 23314
I20260810 19:37:58.533766 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-1/wal/instance
I20260810 19:37:58.534377 30630 external_mini_cluster.cc:1428] Setting key e201e71f0a3393406931a0efe4a0bc1b
I20260810 19:37:58.534408 23383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:58.544682 23383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:37:58 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390678, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:58.571363 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.548280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.29.233.190:35243)
0810 19:37:58.548899 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:58.548907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:58.549902 (+ 995us) server_negotiation.cc:407] Connection header received
0810 19:37:58.552072 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:58.552076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:58.552148 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:58.552258 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:58.554497 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.555302 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.558069 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.558689 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.558811 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.563596 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:58.563602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:58.563606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:58.563648 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:58.566697 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.568034 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.568042 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.568047 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.568248 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.568905 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.568911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.568915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.569523 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:58.569688 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:58.570900 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0810 19:37:58.571089 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":308,"threads_started":1}
I20260810 19:37:58.573271 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.548898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:35243 (local address 127.0.0.1:55308)
0810 19:37:58.549700 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.549767 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:58.551743 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:58.552520 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:58.552534 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:58.553381 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:58.554195 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.554223 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.555510 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.555518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.557831 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.557856 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.558976 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.558988 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.559370 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.560215 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:58.560253 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:58.563294 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:58.566859 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.566884 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.566906 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.567640 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.568428 (+ 788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.568436 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.568441 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.568716 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.569732 (+ 1016us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:58.569745 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:58.570436 (+ 691us) client_negotiation.cc:769] Sending connection context
0810 19:37:58.570814 (+ 378us) client_negotiation.cc:240] Negotiation successful
0810 19:37:58.571444 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for kudu/127.29.233.188@KRBTEST.COM; defaulting to no policy
I20260810 19:37:58.575356 23383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.233.188@KRBTEST.COM" created.
Aug 10 19:37:58 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390678, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:58.591135 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.576339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:52448)
0810 19:37:58.576581 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.576606 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:58.576738 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:58.577824 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:58.577828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:58.578287 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:58.578487 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.578496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.579901 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.579910 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.580955 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.580968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.581793 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.581799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.582000 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.582640 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:58.582658 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:58.585060 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:58.588505 (+ 3445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.588511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.588515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.588891 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.589270 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.589275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.589278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.589422 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.590048 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:58.590054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:58.590200 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 19:37:58.590391 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 19:37:58.590713 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 19:37:58.591231 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.576607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.29.233.189:36395)
0810 19:37:58.577399 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:58.577405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:58.577431 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:37:58.577476 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:58.577482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:58.577664 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:58.577840 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:58.578662 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.579760 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.581089 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.581641 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.581773 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.585284 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:58.585305 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:58.585319 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:58.585355 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:58.588340 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.589026 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.589038 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.589040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.589158 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.589535 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.589540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.589543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.589900 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:58.590124 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:58.590559 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 19:37:58.590831 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":632,"thread_start_us":487,"threads_started":1}
I20260810 19:37:58.593868 23383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:58.595064 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.594599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33824)
0810 19:37:58.594772 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.594893 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 19:37:58.595355 23331 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.597057 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.596699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33830)
0810 19:37:58.596849 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.596941 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 19:37:58.597340 23383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.233.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab.
Entry for principal kudu/127.29.233.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab.
I20260810 19:37:58.622720 30630 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
/tmp/dist-test-taskW7W1IE/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:35821
--webserver_interface=127.29.233.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab
--principal=kudu/127.29.233.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.233.148:36811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.233.190:35243,127.29.233.189:36395,127.29.233.188:35821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 19:37:58.646728 23383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 1
I20260810 19:37:58.648116 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.647471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33842)
0810 19:37:58.647737 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.647886 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 19:37:58.649377 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.648862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33856)
0810 19:37:58.649087 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.649203 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 19:37:58.649645 23383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.697268 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } attempt: 5
Aug 10 19:37:58 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:58.712860 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.698186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.29.233.189:36395)
0810 19:37:58.698405 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:58.698409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:58.698493 (+ 84us) server_negotiation.cc:407] Connection header received
0810 19:37:58.698700 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:58.698704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:58.698772 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:58.698860 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:58.699899 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.700629 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.702469 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.702916 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.703020 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.707248 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:58.707254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:58.707257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:58.707293 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:58.710151 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.710993 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.710997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.710999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.711121 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:58.711539 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:58.711542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:58.711544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:58.711825 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:58.711940 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:58.712451 (+ 511us) server_negotiation.cc:299] Negotiation successful
0810 19:37:58.712628 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 19:37:58.713042 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.698101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:52450)
0810 19:37:58.698380 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.698405 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:58.698562 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:58.698943 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:58.698948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:58.699495 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:58.699722 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.699736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.700774 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.700780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.702215 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:58.702233 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:58.703072 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:58.703079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:58.703292 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:58.703978 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:58.704001 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:58.707021 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:58.710339 (+ 3318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.710346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.710349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.710871 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.711232 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:58.711237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:58.711240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:58.711428 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:58.711946 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:58.711951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:58.712154 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 19:37:58.712319 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 19:37:58.712658 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 19:37:58.714164 23307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:58.715269 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.714805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33868)
0810 19:37:58.714994 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.715095 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 19:37:58.716486 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.716025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33876)
0810 19:37:58.716217 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.716345 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 19:37:58.716729 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.717883 23383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 2
I20260810 19:37:58.719007 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.718470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33888)
0810 19:37:58.718684 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.718798 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 19:37:58.720163 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.719719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33890)
0810 19:37:58.719897 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.719999 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 19:37:58.720430 23383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.767992 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 1
I20260810 19:37:58.769407 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.768834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33906)
0810 19:37:58.769079 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.769200 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 19:37:58.770496 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.770084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33920)
0810 19:37:58.770231 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.770359 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 19:37:58.770721 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.811703 23383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 3
I20260810 19:37:58.813053 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.812469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33924)
0810 19:37:58.812728 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.812846 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 19:37:58.814260 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.813759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33932)
0810 19:37:58.814002 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.814099 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260810 19:37:58.814643 23383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.838994 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 2
I20260810 19:37:58.840528 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.839788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33948)
0810 19:37:58.840104 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.840243 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 19:37:58.841921 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.841431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33964)
0810 19:37:58.841640 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.841751 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 19:37:58.842195 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.948516 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 3
I20260810 19:37:58.949863 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.949229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33966)
0810 19:37:58.949496 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.949625 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 19:37:58.951048 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.950606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33982)
0810 19:37:58.950789 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.950891 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 19:37:58.951274 23307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
I20260810 19:37:58.957883 23383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 4
I20260810 19:37:58.959192 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.958620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33992)
0810 19:37:58.958853 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.958975 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 19:37:58.960359 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:58.959965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:33994)
0810 19:37:58.960141 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:58.960232 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 19:37:58.960589 23383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.233.188:35821: Network error: Client connection negotiation failed: client connection to 127.29.233.188:35821: connect: Connection refused (error 111)
W20260810 19:37:58.993742 23391 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 19:37:58.994077 23391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 19:37:58.994163 23391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 19:37:58.994246 23391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 19:37:59.002045 23391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 19:37:59.002146 23391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 19:37:59.002207 23391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 19:37:59.002262 23391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 19:37:59.002378 23391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 19:37:59.012817 23391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.233.148:36811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.233.190:35243,127.29.233.189:36395,127.29.233.188:35821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/krb5kdc/kudu_127.29.233.188.keytab
--principal=kudu/127.29.233.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.233.188:35821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.233.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 18:32:41 UTC on ccycloud.quasar-qrrker.root.comops.site
build id 4578684
ASAN enabled
I20260810 19:37:59.014179 23391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 19:37:59.015571 23391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 19:37:59.028228 23396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:59.028177 23397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 19:37:59.028245 23399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 19:37:59.029421 23391 server_base.cc:1034] running on GCE node
I20260810 19:37:59.030205 23391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 19:37:59.031766 23391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 19:37:59.033021 23391 hybrid_clock.cc:648] HybridClock initialized: now 1786390679032930 us; error 55 us; skew 500 ppm
I20260810 19:37:59.033408 23391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390679, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 19:37:59.036863 23391 init.cc:377] Logged in from keytab as kudu/127.29.233.188@KRBTEST.COM (short username kudu)
I20260810 19:37:59.038692 23391 webserver.cc:460] Webserver started at http://127.29.233.188:38083/ using document root <none> and password file <none>
I20260810 19:37:59.039453 23391 fs_manager.cc:359] Metadata directory not provided
I20260810 19:37:59.039573 23391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 19:37:59.039913 23391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 19:37:59.043145 23391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data/instance:
uuid: "858d2499c51c4630b4cadd630c8de530"
format_stamp: "Formatted at 2026-08-10 19:37:59 on dist-test-slave-tm1g"
server_key: "4acd9376d1084611d06b2fc6fb4a0093"
server_key_iv: "6f02b8c6bc05cdbf17d15ad11c268a96"
server_key_version: "encryptionkey@0"
I20260810 19:37:59.044013 23391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/wal/instance:
uuid: "858d2499c51c4630b4cadd630c8de530"
format_stamp: "Formatted at 2026-08-10 19:37:59 on dist-test-slave-tm1g"
server_key: "4acd9376d1084611d06b2fc6fb4a0093"
server_key_iv: "6f02b8c6bc05cdbf17d15ad11c268a96"
server_key_version: "encryptionkey@0"
I20260810 19:37:59.049513 23391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 19:37:59.053159 23408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 19:37:59.054632 23391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 19:37:59.054816 23391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data,/tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/wal
uuid: "858d2499c51c4630b4cadd630c8de530"
format_stamp: "Formatted at 2026-08-10 19:37:59 on dist-test-slave-tm1g"
server_key: "4acd9376d1084611d06b2fc6fb4a0093"
server_key_iv: "6f02b8c6bc05cdbf17d15ad11c268a96"
server_key_version: "encryptionkey@0"
I20260810 19:37:59.055054 23391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 19:37:59.080397 23391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 19:37:59.084482 23391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 19:37:59.084758 23391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 19:37:59.103752 23391 rpc_server.cc:307] RPC server started. Bound to: 127.29.233.188:35821
I20260810 19:37:59.103773 23467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.233.188:35821 every 8 connection(s)
I20260810 19:37:59.105134 23391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/data/info.pb
I20260810 19:37:59.108906 23468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 19:37:59.110754 30630 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu as pid 23391
I20260810 19:37:59.110973 30630 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW7W1IE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786390305122638-30630-0/minicluster-data/master-2/wal/instance
I20260810 19:37:59.111598 30630 external_mini_cluster.cc:1428] Setting key 60e7b95cfb226c3bfa4105ecd1602ab9
I20260810 19:37:59.118942 23468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
I20260810 19:37:59.128492 23307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 4
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390679, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.190@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
I20260810 19:37:59.137691 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.113521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.29.233.188:35821)
0810 19:37:59.114435 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.114447 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.114492 (+ 45us) server_negotiation.cc:407] Connection header received
0810 19:37:59.116063 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.116085 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.116505 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.116814 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.118690 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.120105 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.121774 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.122959 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.123150 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.127793 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.127800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.127814 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.127854 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.132836 (+ 4982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.133612 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.133622 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.133632 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.133847 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.134250 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.134260 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.134263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.134989 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.135308 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.135869 (+ 561us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.136311 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":406,"threads_started":1}
I20260810 19:37:59.140642 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.121747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.190:35243 (local address 127.0.0.1:55310)
0810 19:37:59.122628 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:59.122669 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:59.122838 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:59.123344 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:59.123351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:59.124232 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:59.124592 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.124608 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.125658 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.125665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.127061 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.127073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.127841 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.127847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.128048 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.128717 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:59.128753 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:59.132366 (+ 3613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:59.137151 (+ 4785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.137163 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.137169 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.137801 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.138255 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.138260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.138264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.138454 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.139222 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:59.139230 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:59.139512 (+ 282us) client_negotiation.cc:769] Sending connection context
0810 19:37:59.139723 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 19:37:59.140089 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":479,"threads_started":1}
I20260810 19:37:59.140956 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.121726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.29.233.190:35243)
0810 19:37:59.122297 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.122327 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.122760 (+ 433us) server_negotiation.cc:407] Connection header received
0810 19:37:59.123030 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.123039 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.123132 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.123242 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.124781 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.125495 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.127219 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.127638 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.127781 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.132593 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.132601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.132606 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.132645 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.136988 (+ 4343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.137960 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.137965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.137969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.138125 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.138589 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.138593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.138597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.138985 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.139134 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.140230 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.140511 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":254,"threads_started":1}
I20260810 19:37:59.142552 23468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:59.145763 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.129118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:34010)
0810 19:37:59.129332 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:59.129355 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:59.129478 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:59.130179 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:59.130187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:59.130639 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:59.130871 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.130883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.132199 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.132205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.133722 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.133740 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.134781 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.134788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.135018 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.135612 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:59.135636 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:59.138483 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:59.142827 (+ 4344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.142832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.142836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.143270 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.143720 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.143725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.143731 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.143948 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.144600 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:59.144608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:59.144800 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 19:37:59.144952 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 19:37:59.145250 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 19:37:59.145792 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.129196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.29.233.188:35821)
0810 19:37:59.129867 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.129875 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.129903 (+ 28us) server_negotiation.cc:407] Connection header received
0810 19:37:59.129967 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.129970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.130022 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.130144 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.131045 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.132060 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.133899 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.134622 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.134753 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.138758 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.138765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.138769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.138802 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.142634 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.143388 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.143395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.143398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.143578 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.144107 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.144112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.144115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.144470 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.144659 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.145047 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.145343 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":511,"spinlock_wait_cycles":50304,"thread_start_us":311,"threads_started":1}
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.190@KRBTEST.COM
I20260810 19:37:59.151907 23307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d: Bootstrap starting.
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390679, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:59.156293 23307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:59.157780 23307 log.cc:824] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:59.158914 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.141409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.29.233.190:35243)
0810 19:37:59.141571 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.141575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.141889 (+ 314us) server_negotiation.cc:407] Connection header received
0810 19:37:59.141942 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.141946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.142038 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.142152 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.143824 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.144578 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.146500 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.147138 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.147850 (+ 712us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.152212 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.152219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.152223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.152263 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.155842 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.156566 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.156570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.156572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.156722 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.157172 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.157178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.157182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.157567 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.157856 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.158242 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.158499 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:37:59.160043 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.143839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52452 (local address 127.29.233.189:36395)
0810 19:37:59.144061 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.144067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.144087 (+ 20us) server_negotiation.cc:407] Connection header received
0810 19:37:59.144140 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.144146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.144232 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.144325 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.145459 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.146520 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.148531 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.149176 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.149303 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.153049 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.153056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.153066 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.153102 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.156628 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.157418 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.157433 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.157439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.157619 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.158109 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.158114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.158117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.158584 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.158744 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.159394 (+ 650us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.159658 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 19:37:59.160166 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.143427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.189:36395 (local address 127.0.0.1:52452)
0810 19:37:59.143601 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:59.143620 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:59.143707 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:59.144498 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:59.144505 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:59.145001 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:59.145246 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.145257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.146973 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.146979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.148365 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.148381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.149361 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.149372 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.149595 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.150190 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:59.150220 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:59.152839 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:59.156776 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.156783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.156788 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.157287 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.157774 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.157780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.157784 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.157966 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.158874 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:59.158881 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:59.159101 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 19:37:59.159287 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 19:37:59.159658 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":23}
I20260810 19:37:59.162091 23468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } has no permanent_uuid. Determining permanent_uuid...
I20260810 19:37:59.163844 23307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d: No bootstrap required, opened a new log
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390679, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.188@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.233.189@KRBTEST.COM
I20260810 19:37:59.173746 23307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
I20260810 19:37:59.174190 23307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c34ee3175d8481c9c91f0b798ece62d, State: Initialized, Role: FOLLOWER
I20260810 19:37:59.174824 23307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
I20260810 19:37:59.175992 23476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.176556 23476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:59.177384 23307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:59.178287 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.162885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.29.233.188:35821)
0810 19:37:59.163039 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.163043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.163121 (+ 78us) server_negotiation.cc:407] Connection header received
0810 19:37:59.163326 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.163330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.163395 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.163507 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.164505 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.165456 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.166939 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.167363 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.167498 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.171994 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.172003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.172007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.172046 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.174944 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.175759 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.175764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.175768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.175918 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.176375 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.176382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.176387 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.176944 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.177097 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.177667 (+ 570us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.177937 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":39}
I20260810 19:37:59.179234 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.162803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:34024)
0810 19:37:59.163039 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:59.163058 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:59.163207 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:59.163680 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:59.163684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:59.164141 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:59.164337 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.164349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.165577 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.165585 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.166799 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.166811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.167515 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.167520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.167751 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.168325 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:59.168352 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:59.171774 (+ 3422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:59.175078 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.175085 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.175089 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.175625 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.176084 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.176088 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.176091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.176255 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.177187 (+ 932us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:59.177193 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:59.177378 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 19:37:59.177573 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 19:37:59.178971 (+ 1398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":1048}
I20260810 19:37:59.179524 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.161028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52462 (local address 127.29.233.189:36395)
0810 19:37:59.161206 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.161212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.161230 (+ 18us) server_negotiation.cc:407] Connection header received
0810 19:37:59.161292 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.161297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.161373 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.161476 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.162882 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.163787 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.165314 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.165979 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.166105 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.172111 (+ 6006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.172118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.172123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.172165 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.175734 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.176765 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.176771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.176776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.176961 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.177479 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.177484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.177488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.177837 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.178124 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.178512 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.178802 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 19:37:59.184809 30630 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 19:37:59.185124 23468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: Bootstrap starting.
I20260810 19:37:59.188123 23280 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8c34ee3175d8481c9c91f0b798ece62d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55312
I20260810 19:37:59.188509 23280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 19:37:59.188716 23280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:59.189500 23468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:59.191161 23468 log.cc:824] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:59.192741 23280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
I20260810 19:37:59.195472 23280 leader_election.cc:288] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [CANDIDATE]: Term 1 election: Requested vote from peers bc6270d168de4c748af1793b6b2fef63 (127.29.233.189:36395), 858d2499c51c4630b4cadd630c8de530 (127.29.233.188:35821)
W20260810 19:37:59.210660 23255 tablet.cc:2369] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:59.190129 23232 master_runner.cc:186] Error getting master registration for 127.29.233.189:36395: OK, instance_id { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" instance_seqno: 1786390678523619 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 19:37:59.211763 23232 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 19:37:59.216085 23468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: No bootstrap required, opened a new log
W20260810 19:37:59.226436 23489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 19:37:59.226601 23489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 19:37:59.230861 23468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
I20260810 19:37:59.231381 23468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858d2499c51c4630b4cadd630c8de530, State: Initialized, Role: FOLLOWER
I20260810 19:37:59.231957 23468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
I20260810 19:37:59.259795 23383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } attempt: 5
I20260810 19:37:59.259637 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.216824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.29.233.190:35243)
0810 19:37:59.217008 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.217014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.230254 (+ 13240us) server_negotiation.cc:407] Connection header received
0810 19:37:59.230584 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.230591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.230674 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.230746 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.235821 (+ 5075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.237021 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.239134 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.239804 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.239927 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.248926 (+ 8999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.248939 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.248945 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.248992 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.253157 (+ 4165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.255092 (+ 1935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.255098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.255102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.255263 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.256356 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.256362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.256365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.256785 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.257073 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.258390 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.258661 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:37:59.265933 23499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.267737 23441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c34ee3175d8481c9c91f0b798ece62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "858d2499c51c4630b4cadd630c8de530"
I20260810 19:37:59.268021 23499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:59.268350 23441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 19:37:59 dist-test-slave-tm1g krb5kdc[23219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786390678, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.233.189@KRBTEST.COM for kudu/127.29.233.188@KRBTEST.COM
I20260810 19:37:59.267674 23468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:59.272122 23441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c34ee3175d8481c9c91f0b798ece62d in term 1.
I20260810 19:37:59.272861 23249 leader_election.cc:302] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 858d2499c51c4630b4cadd630c8de530, 8c34ee3175d8481c9c91f0b798ece62d; no voters:
I20260810 19:37:59.273423 23476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 19:37:59.274263 23476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [term 1 LEADER]: Becoming Leader. State: Replica: 8c34ee3175d8481c9c91f0b798ece62d, State: Running, Role: LEADER
I20260810 19:37:59.276407 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.216728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52478 (local address 127.29.233.189:36395)
0810 19:37:59.216918 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.216924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.224094 (+ 7170us) server_negotiation.cc:407] Connection header received
0810 19:37:59.225397 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.225405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.225516 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.225598 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.233239 (+ 7641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.234607 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.242917 (+ 8310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.243955 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.244105 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.256155 (+ 12050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.256166 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.256174 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.256240 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.261342 (+ 5102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.265379 (+ 4037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.265425 (+ 46us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.265430 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.265773 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.266867 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.266872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.266910 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.267910 (+ 1000us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.268192 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.275713 (+ 7521us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.275976 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":313,"server-negotiator.queue_time_us":66}
I20260810 19:37:59.278257 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.224682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34030 (local address 127.29.233.188:35821)
0810 19:37:59.224893 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.224900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.231630 (+ 6730us) server_negotiation.cc:407] Connection header received
0810 19:37:59.232165 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.232170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.232291 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.232368 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.245440 (+ 13072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.246967 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.253404 (+ 6437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.254523 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.254663 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.268677 (+ 14014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.268683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.268688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.268735 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.273602 (+ 4867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.275324 (+ 1722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.275329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.275332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.275664 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.276504 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.276512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.276517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.276994 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.277271 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.277667 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.277905 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 19:37:59.280653 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.260602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.233.188:35821 (local address 127.0.0.1:34032)
0810 19:37:59.261013 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 19:37:59.261035 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 19:37:59.261142 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 19:37:59.262072 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 19:37:59.262077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 19:37:59.262602 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0810 19:37:59.262845 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.262857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.264340 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.264345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.265871 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 19:37:59.265885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.267102 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.267110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 19:37:59.267410 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.268057 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 19:37:59.268082 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 19:37:59.270927 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 19:37:59.276943 (+ 6016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.276950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.276953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.277510 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.277998 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 19:37:59.278004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 19:37:59.278008 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 19:37:59.278262 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 19:37:59.279357 (+ 1095us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 19:37:59.279364 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 19:37:59.279578 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 19:37:59.279760 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 19:37:59.280129 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":3}
I20260810 19:37:59.276662 23476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
I20260810 19:37:59.281924 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.261406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.29.233.188:35821)
0810 19:37:59.261574 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.261581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.261604 (+ 23us) server_negotiation.cc:407] Connection header received
0810 19:37:59.261662 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.261668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.261756 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.261876 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.263032 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.264150 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.266035 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.266763 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.266901 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.271155 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 19:37:59.271163 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 19:37:59.271169 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 19:37:59.271215 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 19:37:59.276463 (+ 5248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.277670 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.277675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.277679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.277827 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 19:37:59.278421 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 19:37:59.278426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 19:37:59.278429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 19:37:59.279138 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 19:37:59.279287 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.281061 (+ 1774us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.281320 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 19:37:59.291109 23500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c34ee3175d8481c9c91f0b798ece62d. Latest consensus state: current_term: 1 leader_uuid: "8c34ee3175d8481c9c91f0b798ece62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.291560 23500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:59.291805 23383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63: Bootstrap starting.
I20260810 19:37:59.296007 23383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63: Neither blocks nor log segments found. Creating new log.
I20260810 19:37:59.297497 23383 log.cc:824] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63: Log is configured to *not* fsync() on all Append() calls
I20260810 19:37:59.307298 23383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63: No bootstrap required, opened a new log
I20260810 19:37:59.291355 23391 master_runner.cc:186] Error getting master registration for 127.29.233.189:36395: OK, instance_id { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" instance_seqno: 1786390678523619 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 19:37:59.308138 23391 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 19:37:59.310417 23513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 19:37:59.315591 23513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 19:37:59.319516 23383 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
I20260810 19:37:59.319993 23383 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6270d168de4c748af1793b6b2fef63, State: Initialized, Role: FOLLOWER
I20260810 19:37:59.320667 23383 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } }
W20260810 19:37:59.327029 23511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 19:37:59.327185 23511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 19:37:59.327943 23441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [term 1 FOLLOWER]: Refusing update from remote peer 8c34ee3175d8481c9c91f0b798ece62d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:59.328979 23500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [LEADER]: Connected to new peer: Peer: permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:59.329798 23518 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.330240 23518 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 19:37:59.336850 23417 tablet.cc:2369] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 19:37:59.340241 23383 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 19:37:59.340683 23356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [term 0 FOLLOWER]: Advancing to term 1
I20260810 19:37:59.345495 23356 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [term 1 FOLLOWER]: Refusing update from remote peer 8c34ee3175d8481c9c91f0b798ece62d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 19:37:59.346387 23357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c34ee3175d8481c9c91f0b798ece62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bc6270d168de4c748af1793b6b2fef63"
I20260810 19:37:59.346474 23500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 19:37:59.350085 23357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c34ee3175d8481c9c91f0b798ece62d in term 1.
I20260810 19:37:59.358048 23499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c34ee3175d8481c9c91f0b798ece62d. Latest consensus state: current_term: 1 leader_uuid: "8c34ee3175d8481c9c91f0b798ece62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.358417 23499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:59.363857 23519 mvcc.cc:205] Tried to move back new op lower bound from 7317056222512271360 to 7317056222348107776. Current Snapshot: MvccSnapshot[applied={T|T < 7317056222512271360}]
I20260810 19:37:59.379098 23518 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c34ee3175d8481c9c91f0b798ece62d. Latest consensus state: current_term: 1 leader_uuid: "8c34ee3175d8481c9c91f0b798ece62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.379396 23518 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:59.379527 23499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c34ee3175d8481c9c91f0b798ece62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.379839 23499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:59.384555 23513 catalog_manager.cc:1347] Generated new cluster ID: 590decd44dce47b5b55de79c475999f2
I20260810 19:37:59.384680 23513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 19:37:59.385110 23500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c34ee3175d8481c9c91f0b798ece62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.385413 23500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:59.385735 23500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c34ee3175d8481c9c91f0b798ece62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.385939 23500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d [sys.catalog]: This master's current role is: LEADER
I20260810 19:37:59.386591 23524 mvcc.cc:205] Tried to move back new op lower bound from 7317056222512271360 to 7317056222348107776. Current Snapshot: MvccSnapshot[applied={T|T < 7317056222512271360}]
I20260810 19:37:59.391880 23518 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c34ee3175d8481c9c91f0b798ece62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c34ee3175d8481c9c91f0b798ece62d" member_type: VOTER last_known_addr { host: "127.29.233.190" port: 35243 } } peers { permanent_uuid: "bc6270d168de4c748af1793b6b2fef63" member_type: VOTER last_known_addr { host: "127.29.233.189" port: 36395 } } peers { permanent_uuid: "858d2499c51c4630b4cadd630c8de530" member_type: VOTER last_known_addr { host: "127.29.233.188" port: 35821 } } }
I20260810 19:37:59.392125 23518 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 19:37:59.404184 23513 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 19:37:59.406340 23513 catalog_manager.cc:1504] Loading token signing keys...
I20260810 19:37:59.440691 23513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c34ee3175d8481c9c91f0b798ece62d: Generated new TSK 0
I20260810 19:37:59.442080 23513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 19:37:59.446089 23537 catalog_manager.cc:1259] Loaded cluster ID: 590decd44dce47b5b55de79c475999f2
I20260810 19:37:59.446259 23537 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63: loading cluster ID for follower catalog manager: success
I20260810 19:37:59.453362 23537 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63: acquiring CA information for follower catalog manager: success
I20260810 19:37:59.456149 23537 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc6270d168de4c748af1793b6b2fef63: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 19:37:59.600915 23472 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.579931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55336 (local address 127.29.233.190:35243)
0810 19:37:59.580503 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.580514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.597908 (+ 17394us) server_negotiation.cc:407] Connection header received
0810 19:37:59.598752 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.598762 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.598879 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.598975 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.600558 (+ 1583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55336: BlockingRecv error: recv got EOF from 127.0.0.1:55336 (error 108)
Metrics: {"server-negotiator.queue_time_us":352}
W20260810 19:37:59.602998 23388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.580702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.29.233.189:36395)
0810 19:37:59.580971 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.580979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.598264 (+ 17285us) server_negotiation.cc:407] Connection header received
0810 19:37:59.601350 (+ 3086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.601358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.601472 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.601555 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.602742 (+ 1187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52492: BlockingRecv error: recv got EOF from 127.0.0.1:52492 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260810 19:37:59.604075 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.597756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.29.233.188:35821)
0810 19:37:59.597985 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.597991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.602168 (+ 4177us) server_negotiation.cc:407] Connection header received
0810 19:37:59.602460 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.602466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.602556 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.602634 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.603810 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34040: BlockingRecv error: recv got EOF from 127.0.0.1:34040 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 19:37:59.660404 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.642057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.29.233.189:36395)
0810 19:37:59.642253 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.642259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.651935 (+ 9676us) server_negotiation.cc:407] Connection header received
0810 19:37:59.652504 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.652510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.652714 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.652862 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:59.653642 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.654876 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.656974 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.657707 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.657805 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.658644 (+ 839us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:59.659374 (+ 730us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:59.659506 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.659940 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.660053 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 19:37:59.713063 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.703461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.29.233.188:35821)
0810 19:37:59.703689 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.703695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.710987 (+ 7292us) server_negotiation.cc:407] Connection header received
0810 19:37:59.711292 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.711300 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.711450 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.711539 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.712801 (+ 1262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34046: BlockingRecv error: recv got EOF from 127.0.0.1:34046 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 19:37:59.766610 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.752523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.29.233.189:36395)
0810 19:37:59.752793 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.752802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.758974 (+ 6172us) server_negotiation.cc:407] Connection header received
0810 19:37:59.759276 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.759283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.759423 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.759526 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:59.760348 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.761546 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.764129 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.764841 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.764947 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.765611 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:59.765799 (+ 188us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:59.765874 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.766120 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.766217 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 19:37:59.822374 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.809122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.29.233.188:35821)
0810 19:37:59.809325 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.809332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.820035 (+ 10703us) server_negotiation.cc:407] Connection header received
0810 19:37:59.820635 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.820642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.820763 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.820852 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.822080 (+ 1228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34058: BlockingRecv error: recv got EOF from 127.0.0.1:34058 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 19:37:59.864923 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.857070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.29.233.189:36395)
0810 19:37:59.857275 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.857281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.857901 (+ 620us) server_negotiation.cc:407] Connection header received
0810 19:37:59.858104 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.858110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.858199 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.858331 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:59.858949 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.860097 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.862185 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.862839 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.862991 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.863976 (+ 985us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:59.864151 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:59.864257 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.864453 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.864552 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 19:37:59.879602 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.872691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.29.233.188:35821)
0810 19:37:59.872891 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.872897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.877650 (+ 4753us) server_negotiation.cc:407] Connection header received
0810 19:37:59.877837 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.877844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.877991 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.878137 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.879284 (+ 1147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34060: BlockingRecv error: recv got EOF from 127.0.0.1:34060 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:37:59.895097 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.887189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52526 (local address 127.29.233.189:36395)
0810 19:37:59.887402 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.887409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.887955 (+ 546us) server_negotiation.cc:407] Connection header received
0810 19:37:59.888177 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.888184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.888305 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.888437 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:59.889001 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.890280 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.892601 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.893492 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.893639 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.894198 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:59.894379 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:59.894489 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.894674 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.894774 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 19:37:59.905067 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.903128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.29.233.188:35821)
0810 19:37:59.903456 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.903475 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.903501 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:37:59.903580 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.903586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.903681 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.903791 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.904777 (+ 986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34066: BlockingRecv error: recv got EOF from 127.0.0.1:34066 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
I20260810 19:37:59.924873 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.917579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52536 (local address 127.29.233.189:36395)
0810 19:37:59.917800 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.917807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.918456 (+ 649us) server_negotiation.cc:407] Connection header received
0810 19:37:59.918706 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.918712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.918814 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.918924 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:59.919457 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.920619 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.922677 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.923370 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.923501 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.924018 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:59.924151 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:59.924247 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.924428 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.924524 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260810 19:37:59.934218 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.931896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.29.233.188:35821)
0810 19:37:59.932099 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.932107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.932615 (+ 508us) server_negotiation.cc:407] Connection header received
0810 19:37:59.932820 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.932826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.932928 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.933038 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.933947 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34082: BlockingRecv error: recv got EOF from 127.0.0.1:34082 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 19:37:59.962011 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:37:59.954435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.29.233.189:36395)
0810 19:37:59.954653 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.954659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.955237 (+ 578us) server_negotiation.cc:407] Connection header received
0810 19:37:59.955559 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.955566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.955666 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.955736 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:37:59.956416 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.957665 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.959852 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:37:59.960492 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:37:59.960605 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:37:59.961217 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:37:59.961352 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:37:59.961442 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 19:37:59.961628 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 19:37:59.961721 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260810 19:37:59.971441 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:37:59.969343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.29.233.188:35821)
0810 19:37:59.969560 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 19:37:59.969566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:37:59.969892 (+ 326us) server_negotiation.cc:407] Connection header received
0810 19:37:59.970099 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:37:59.970103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:37:59.970176 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:37:59.970259 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:37:59.971190 (+ 931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34092: BlockingRecv error: recv got EOF from 127.0.0.1:34092 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 19:38:00.013918 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.007472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52554 (local address 127.29.233.189:36395)
0810 19:38:00.007797 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.007804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.008114 (+ 310us) server_negotiation.cc:407] Connection header received
0810 19:38:00.008327 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.008331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.008405 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.008513 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.009061 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.009885 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.012003 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.012512 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.012619 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.013132 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.013265 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.013336 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.013528 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.013609 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260810 19:38:00.022344 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:38:00.020274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.29.233.188:35821)
0810 19:38:00.020488 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.020494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.020799 (+ 305us) server_negotiation.cc:407] Connection header received
0810 19:38:00.020984 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.020989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.021069 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.021153 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:38:00.022073 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34096: BlockingRecv error: recv got EOF from 127.0.0.1:34096 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 19:38:00.095075 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.089749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.29.233.189:36395)
0810 19:38:00.089922 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.089926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.090355 (+ 429us) server_negotiation.cc:407] Connection header received
0810 19:38:00.090583 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.090587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.090663 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.090756 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.091213 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.091981 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.093495 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.093926 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.094023 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.094472 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.094585 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.094658 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.094795 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.094861 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 19:38:00.102694 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:38:00.100799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.29.233.188:35821)
0810 19:38:00.101090 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.101097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.101361 (+ 264us) server_negotiation.cc:407] Connection header received
0810 19:38:00.101529 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.101535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.101617 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.101703 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:38:00.102391 (+ 688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34106: BlockingRecv error: recv got EOF from 127.0.0.1:34106 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 19:38:00.241719 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.236240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.29.233.189:36395)
0810 19:38:00.236450 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.236455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.236955 (+ 500us) server_negotiation.cc:407] Connection header received
0810 19:38:00.237141 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.237146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.237223 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.237315 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.237744 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.238529 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.239985 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.240534 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.240647 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.241057 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.241195 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.241265 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.241382 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.241456 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 19:38:00.249866 23471 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 19:38:00.247998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34108 (local address 127.29.233.188:35821)
0810 19:38:00.248172 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.248177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.248606 (+ 429us) server_negotiation.cc:407] Connection header received
0810 19:38:00.248764 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.248768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.248842 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.248931 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 19:38:00.249678 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34108: BlockingRecv error: recv got EOF from 127.0.0.1:34108 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 19:38:00.329275 23511 catalog_manager.cc:1259] Loaded cluster ID: 590decd44dce47b5b55de79c475999f2
I20260810 19:38:00.329399 23511 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: loading cluster ID for follower catalog manager: success
I20260810 19:38:00.332118 23511 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: acquiring CA information for follower catalog manager: success
I20260810 19:38:00.334035 23511 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 858d2499c51c4630b4cadd630c8de530: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 19:38:00.517372 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.512276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.29.233.189:36395)
0810 19:38:00.512462 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.512466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.512754 (+ 288us) server_negotiation.cc:407] Connection header received
0810 19:38:00.512925 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.512930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.513010 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.513102 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.513465 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.514222 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.515659 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.516075 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.516214 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.516651 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.516777 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.516875 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.517031 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.517110 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 19:38:00.529395 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.523537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34114 (local address 127.29.233.188:35821)
0810 19:38:00.523756 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.523763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.523955 (+ 192us) server_negotiation.cc:407] Connection header received
0810 19:38:00.524153 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.524157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.524314 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.524433 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.524838 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.525638 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.527001 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.527391 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.527517 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.528013 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.528729 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.528901 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.528996 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.529080 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 19:38:00.547624 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.535687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.29.233.189:36395)
0810 19:38:00.535934 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.535943 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.540699 (+ 4756us) server_negotiation.cc:407] Connection header received
0810 19:38:00.541064 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.541070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.541163 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.541248 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.542360 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.543243 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.545365 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.545999 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.546128 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.546711 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.546875 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.546967 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.547170 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.547266 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 19:38:00.547868 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.540079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.29.233.188:35821)
0810 19:38:00.540276 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.540283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.541334 (+ 1051us) server_negotiation.cc:407] Connection header received
0810 19:38:00.541512 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.541517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.541607 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.541706 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.542764 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.543821 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.545761 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.546415 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.546546 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.547040 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.547198 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.547282 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.547501 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.547577 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 19:38:00.549373 23666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 19:38:00.536168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.29.233.190:35243)
0810 19:38:00.540908 (+ 4740us) server_negotiation.cc:206] Beginning negotiation
0810 19:38:00.540916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 19:38:00.540942 (+ 26us) server_negotiation.cc:407] Connection header received
0810 19:38:00.541075 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 19:38:00.541082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 19:38:00.541320 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 19:38:00.541512 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 19:38:00.542653 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.543813 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.545968 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 19:38:00.546614 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 19:38:00.546757 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 19:38:00.547307 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 19:38:00.547848 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 19:38:00.547977 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 19:38:00.549075 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0810 19:38:00.549156 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4604,"thread_start_us":4551,"threads_started":1}
I20260810 19:38:00.554472 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 23232
I20260810 19:38:00.555446 23302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:38:00.672983 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 23232
I20260810 19:38:00.707197 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 23314
I20260810 19:38:00.708595 23377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:38:00.818634 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 23314
I20260810 19:38:00.843176 30630 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu pid 23391
I20260810 19:38:00.844278 23463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 19:38:00.953318 30630 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW7W1IE/build/asan/bin/kudu with pid 23391
2026-08-10T19:38:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5004 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10376 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (375795 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST